Data Fetching Guide

The shared.data_fetching module provides structured clients for accessing external APIs and data sources.

Academic & Research Clients

arXiv Client

Module: arxiv_client.py No API Key

from shared.data_fetching.arxiv_client import ArxivClient
client = ArxivClient()
results = client.search(query='LLM agents', max_results=10)

Semantic Scholar Client

Module: semantic_scholar_client.py No API Key

from shared.data_fetching.semantic_scholar_client import SemanticScholarClient
client = SemanticScholarClient()
papers = client.search_paper('Attention is All You Need', limit=1)

Technical Clients

GitHub Client

Module: github_client.py API Key Required

from shared.data_fetching.github_client import GitHubClient
client = GitHubClient(api_key='ghp_...')
repos = client.search_repositories('language:python stars:>1000')

Science & Reference

NASA Client

Module: nasa_client.py Optional Key

from shared.data_fetching.nasa_client import NASAClient
client = NASAClient()
apod = client.get_apod()

Wikipedia Client

Module: wikipedia_client.py No API Key

from shared.data_fetching.wikipedia_client import WikipediaClient
client = WikipediaClient()
summary = client.get_summary('Artificial_intelligence')

Factory Pattern

You can use the factory to create any client dynamically:

from shared.data_fetching import DataFetchingFactory

# List available sources
sources = DataFetchingFactory.list_sources()

# Create a client
client = DataFetchingFactory.create_client('weather')

Caching

All clients support caching to minimize API calls and latency.

Last Updated: November 19, 2025