Commit Graph

52 Commits

Author SHA1 Message Date
Piyush Tripathi
71d5d71a80 test(fetch): add unit tests for fetch MCP server
Add comprehensive test coverage for the fetch server:
- TestGetRobotsTxtUrl: 6 tests for URL parsing
- TestExtractContentFromHtml: 3 tests for HTML-to-markdown conversion
- TestCheckMayAutonomouslyFetchUrl: 5 tests for robots.txt handling
- TestFetchUrl: 6 tests for URL fetching with various scenarios

Total: 20 tests covering:
- URL parsing and robots.txt URL generation
- HTML content extraction and markdown conversion
- robots.txt permission checking (401, 403, 404, allow/disallow)
- HTTP response handling (success, errors, raw mode)
- Proxy support

Also adds pytest and pytest-asyncio as dev dependencies.
2026-01-28 14:31:01 -08:00
dependabot[bot]
a1f9dfa470 Bump the uv group across 3 directories with 1 update
Bumps the uv group with 1 update in the /src/fetch directory: [python-multipart](https://github.com/Kludex/python-multipart).
Bumps the uv group with 1 update in the /src/git directory: [python-multipart](https://github.com/Kludex/python-multipart).
Bumps the uv group with 1 update in the /src/time directory: [python-multipart](https://github.com/Kludex/python-multipart).


Updates `python-multipart` from 0.0.21 to 0.0.22
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.21...0.0.22)

Updates `python-multipart` from 0.0.21 to 0.0.22
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.21...0.0.22)

Updates `python-multipart` from 0.0.21 to 0.0.22
- [Release notes](https://github.com/Kludex/python-multipart/releases)
- [Changelog](https://github.com/Kludex/python-multipart/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Kludex/python-multipart/compare/0.0.21...0.0.22)

---
updated-dependencies:
- dependency-name: python-multipart
  dependency-version: 0.0.22
  dependency-type: indirect
  dependency-group: uv
- dependency-name: python-multipart
  dependency-version: 0.0.22
  dependency-type: indirect
  dependency-group: uv
- dependency-name: python-multipart
  dependency-version: 0.0.22
  dependency-type: indirect
  dependency-group: uv
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-26 23:57:45 +00:00
dependabot[bot]
b906f726fa Bump the uv group across 3 directories with 5 updates
Bumps the uv group with 5 updates in the /src/fetch directory:

| Package | From | To |
| --- | --- | --- |
| [mcp](https://github.com/modelcontextprotocol/python-sdk) | `1.2.0` | `1.23.0` |
| [requests](https://github.com/psf/requests) | `2.32.3` | `2.32.4` |
| [h11](https://github.com/python-hyper/h11) | `0.14.0` | `0.16.0` |
| [starlette](https://github.com/Kludex/starlette) | `0.41.2` | `0.49.1` |
| [urllib3](https://github.com/urllib3/urllib3) | `2.2.3` | `2.6.3` |

Bumps the uv group with 3 updates in the /src/git directory: [mcp](https://github.com/modelcontextprotocol/python-sdk), [h11](https://github.com/python-hyper/h11) and [starlette](https://github.com/Kludex/starlette).
Bumps the uv group with 2 updates in the /src/time directory: [h11](https://github.com/python-hyper/h11) and [starlette](https://github.com/Kludex/starlette).


Updates `mcp` from 1.2.0 to 1.23.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.2.0...v1.23.0)

Updates `requests` from 2.32.3 to 2.32.4
- [Release notes](https://github.com/psf/requests/releases)
- [Changelog](https://github.com/psf/requests/blob/main/HISTORY.md)
- [Commits](https://github.com/psf/requests/compare/v2.32.3...v2.32.4)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.2 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `urllib3` from 2.2.3 to 2.6.3
- [Release notes](https://github.com/urllib3/urllib3/releases)
- [Changelog](https://github.com/urllib3/urllib3/blob/main/CHANGES.rst)
- [Commits](https://github.com/urllib3/urllib3/compare/2.2.3...2.6.3)

Updates `mcp` from 1.2.0 to 1.23.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.2.0...v1.23.0)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.2 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `mcp` from 1.2.0 to 1.23.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.2.0...v1.23.0)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.2 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `mcp` from 1.1.0 to 1.23.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.2.0...v1.23.0)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.3 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `mcp` from 1.1.0 to 1.23.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.2.0...v1.23.0)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.3 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `mcp` from 1.1.0 to 1.23.0
- [Release notes](https://github.com/modelcontextprotocol/python-sdk/releases)
- [Changelog](https://github.com/modelcontextprotocol/python-sdk/blob/main/RELEASE.md)
- [Commits](https://github.com/modelcontextprotocol/python-sdk/compare/v1.2.0...v1.23.0)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.3 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.3 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.3 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

Updates `h11` from 0.14.0 to 0.16.0
- [Commits](https://github.com/python-hyper/h11/compare/v0.14.0...v0.16.0)

Updates `starlette` from 0.41.3 to 0.49.1
- [Release notes](https://github.com/Kludex/starlette/releases)
- [Changelog](https://github.com/Kludex/starlette/blob/main/docs/release-notes.md)
- [Commits](https://github.com/Kludex/starlette/compare/0.41.2...0.49.1)

---
updated-dependencies:
- dependency-name: mcp
  dependency-version: 1.23.0
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: requests
  dependency-version: 2.32.4
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: urllib3
  dependency-version: 2.6.3
  dependency-type: indirect
  dependency-group: uv
- dependency-name: mcp
  dependency-version: 1.23.0
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: mcp
  dependency-version: 1.23.0
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: mcp
  dependency-version: 1.23.0
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: mcp
  dependency-version: 1.23.0
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: mcp
  dependency-version: 1.23.0
  dependency-type: direct:production
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
- dependency-name: h11
  dependency-version: 0.16.0
  dependency-type: indirect
  dependency-group: uv
- dependency-name: starlette
  dependency-version: 0.49.1
  dependency-type: indirect
  dependency-group: uv
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-01-17 18:03:32 +00:00
adam jones
d9929014d5 Add mcpName and repository fields for MCP registry (#3068)
Adds metadata required for registering servers in the MCP registry:

- NPM servers (everything, filesystem, memory, sequentialthinking):
  Added mcpName and repository fields to package.json

- PyPI servers (fetch, git, time):
  Added mcp-name comment to README.md

Part of #3047
2025-11-26 14:29:36 +00:00
Adam Jones
78e0088ebd fix(fetch): update uv.lock to sync with pyproject.toml
The lockfile was out of sync because pyproject.toml was updated
(version bump to 0.6.3, httpx<0.28 constraint) but uv.lock wasn't
regenerated. This became a CI failure after switching from --frozen
to --locked in #1852.
2025-11-25 21:52:09 +00:00
Eugene Mwangi
9758f34d36 chore(uv): Prefer --locked for dependency sync (#1852)
Adopt uv best practice for dependency synchronization.

[Note]
- (uv sync --frozen) proceeds with an out-dated lockfile.
  This could lead to inconsistencies.

[Docs]
See: https://docs.astral.sh/uv/concepts/projects/sync/#locking-and-syncing
2025-11-25 21:36:35 +00:00
claude[bot]
3dd0db07fd docs(fetch): Add Windows PYTHONIOENCODING configuration
Add troubleshooting section for Windows users experiencing timeout issues.
Setting PYTHONIOENCODING=utf-8 fixes character encoding problems that
can cause the server to timeout on Windows systems.

Fixes #65

Co-authored-by: Ola Hungerford <olaservo@users.noreply.github.com>
2025-08-17 16:46:33 +00:00
Tadas Antanavicius
0e519ca3a1 Merge pull request #1519 from olaservo/olaservo-patch-1
Update fetch Readme JSON snippets for installation to be complete JSON
2025-05-31 11:27:34 -07:00
David Dworken
c0821f1ed8 Add warning about security risks of 2 MCP servers (#1765) 2025-05-15 17:49:25 +01:00
Ola Hungerford
5514774490 Update fetch Readme JSON snippets for installation to be complete JSON 2025-04-17 19:52:45 -07:00
Burke Holland
090b6b7c1a Move VS Code below Claude 2025-04-10 12:10:02 -05:00
Burke Holland
76b3d7b5de Merge remote-tracking branch 'upstream/main' into burkeholland-vscode-install-instructions 2025-04-10 09:29:23 -05:00
Burke Holland
e6db372447 Update READMEs with instructions for VS Code installation 2025-04-03 16:37:27 -05:00
Ola Hungerford
8f73c9ac4c Merge branch 'main' into main 2025-03-30 08:32:53 -07:00
shiquda
9a4d513724 fix(fetch): specify httpx<0.28 to resolve proxy problem
- AsyncClient.__init__() got an unexpected keyword argument 'proxies'
2025-03-30 22:33:51 +08:00
Wilhelm Klopp
0490e08253 Fix typo in mcp_server_fetch 2025-03-26 18:52:33 +00:00
shiquda
64a654744a feat(fetch): add support for using proxy for requests 2025-03-13 00:20:48 +08:00
David Soria Parra
53f4a23b73 Update server implementation and dependencies 2025-01-15 17:27:12 +00:00
Aleksey Zhukov
ebecd590e2 Update error handling to use ErrorData consistently
Also:
Updated mcp dependency from 1.0.0 to 1.1.3

This aligns with MCP's error handling patterns and provides better
structured error information.
2025-01-10 21:53:57 +01:00
yangdx
ce72760302 Fix webpage content pagination logic error
Improve truncated content handling. Fix empty content return logic •
2025-01-02 16:51:23 +08:00
Jim Clark
4fa1c47b0a Update fetch Docker to not have ux in final stage 2024-12-19 16:05:22 -08:00
Jim Clark
70e19c466d Migrate python servers to mcp namespace 2024-12-19 13:42:46 -08:00
Jim Clark
368e3b23ca Add Dockerfiles for the 17 sample MCP servers
* add Dockerfiles and update README.md definitions
2024-12-19 13:11:34 -08:00
David Soria Parra
e7e1c85058 python servers 0.6.2 2024-12-04 15:57:31 +00:00
Justin Spahr-Summers
d1650ce5fb Merge pull request #144 from SkywaveTM/main
fix(fetch): properly handles robots.txt
2024-12-03 07:18:33 -06:00
Jack Adamson
45dfd82192 fix deserialization of URL 2024-12-02 13:09:10 +00:00
YeongJun
d5bae8759f follows redirects on checking robots.txt 2024-11-30 23:31:30 +09:00
YeongJun
168522f34d ensure url is string type in robot_parser.can_fetch() 2024-11-30 23:10:12 +09:00
David Soria Parra
2578d6f668 python servers 0.6.1 2024-11-29 18:00:23 +00:00
David Soria Parra
f86486984c python servers 0.6.0 2024-11-29 17:54:40 +00:00
Jack Adamson
bee382cec5 make changes requested in PR 2024-11-29 15:45:10 +00:00
Jack Adamson
a9e37d25c6 switch pydantic type annotations to satisfy pyright 2024-11-29 15:22:46 +00:00
Jack Adamson
e0234c7212 Merge pull request #130 from modelcontextprotocol/jadamson/fetch-use-readabilityjs
feat(fetch): add fetching of raw text, pagination and keeping links in the markdown
2024-11-29 15:09:54 +00:00
Jack Adamson
ea42a21078 add doc strings for readabilty and constrain types 2024-11-29 14:54:06 +00:00
Jack Adamson
c820086b35 update README to reflect new capabilities 2024-11-29 14:45:57 +00:00
Jack Adamson
5552af104c format with black 2024-11-29 13:37:35 +00:00
Jack Adamson
b6710dae1a add argument to fetch raw html 2024-11-29 13:16:09 +00:00
Jack Adamson
e8dcd29427 add pagination of fetches so models can avoid reading a full page if it's got the information it needs 2024-11-29 13:04:16 +00:00
Jack Adamson
960321f136 improve error message to model on fetch failure 2024-11-29 12:23:18 +00:00
Jack Adamson
37622d3872 add handling of non-html pages 2024-11-29 11:51:41 +00:00
anjor
39d6a3bc6b fix zed config 2024-11-28 22:01:38 +00:00
Jack Adamson
467330de6e update fetch server to use readability JS if node is installed 2024-11-28 18:44:33 +00:00
Ikko Eltociear Ashimine
b16a541c2e chore: update server.py
minor fix
2024-11-28 14:35:52 +09:00
Jack Adamson
15c9c6f4b1 remove redundant gitignore 2024-11-25 20:05:10 +00:00
Jack Adamson
caed64d5db update dependencies 2024-11-25 20:04:59 +00:00
Jack Adamson
fe9777401f update README to document commandline arguments 2024-11-25 19:38:23 +00:00
Jack Adamson
51af6b56ef add command-line arguments for controlling user-agent and whether to obey robots.txt 2024-11-25 19:31:23 +00:00
Jack Adamson
84321c89cf make the fetch mcp-server obey robots.txt 2024-11-22 19:04:04 +00:00
Jack Adamson
c08d2a3cf6 fix lint 2024-11-22 19:03:43 +00:00
Jack Adamson
20a03e7395 add python-version file 2024-11-22 18:54:02 +00:00