Enterprise IPFS for the AI Era
The NDN Data Protocol (NDP) v1.0 collapses three separate protocol stacks into one. One canonical envelope spans three database workloads — Blobs, Models, and Structured Records — under a single auth surface, billing meter, and audit trail.
Production IPFS applications today assemble a patchwork from three separate tool families, none of which interoperate at the protocol level.
Pinata, Web3.Storage, Filebase, Fleek standardized blob pinning — a genuine success. But it stops at bytes. No query. No schema enforcement. No model semantics. A team needing structured data or model weights on top of blob storage is on their own.
⚠ A dApp developer pinning NFT metadata with Pinata and user records with Postgres has no single content-addressed audit trail.
HuggingFace Hub is centralized. S3 gives no CID. A 70B-parameter model consists of hundreds of weight shards, a shard map, a model card, and eval results — no standard content-addressed format exists for this. Reproducibility fails at the storage layer.
⚠ Teams must write bespoke download-and-pin scripts, manage shard ordering, and construct shard maps by hand.
Tableland offers SQL-flavored data with no blob or model handling. Ceramic gives mutable documents with heavy client complexity. OrbitDB is embedded-only. None expose a standard content-addressed envelope that produces the same CID across two independent implementations.
⚠ Compliance teams cannot prove "this exact dataset was in this exact state at this timestamp" with a standard primitive.
The NDN Data Protocol makes one claim: every persistent object a production application needs can be expressed as a canonical JSON envelope, hashed deterministically, and addressed by a CIDv1. Three data planes emerge naturally from this foundation.
The parent field forms a content-addressed version chain. The tenant field is part of the canonical form — two tenants storing identical content produce different CIDs, ensuring multi-tenant isolation by design.
All three data planes share one tenant-scoped authentication model, one billing meter, one audit trail, and AES-256-GCM envelope encryption with KMS-managed keys. A single GET /v1/_discovery endpoint advertises which data planes a deployment exposes.
_audit collection. Tamper-evident.
GET /v1/_discovery advertises which data planes are live on any given deployment.
Each data plane has its own API surface and semantics. A deployment MAY implement all three or declare partial support via GET /v1/_discovery.
Arbitrary-byte payloads. A strict superset of IPFS Pinning Services API v1.0 — every existing tool works without modification.
Content-addressed AI/ML model registry. Each version is a UnixFS DAG: weight shards + shard map + model card + version envelope.
Queryable JSON documents with schemas, immutable version chains, collections, and content-addressed views.
/v1/blobs · /v1/pins · /v1/upload
| Method | Path | Purpose |
|---|---|---|
| POST | /v1/blobs | Upload raw bytes. Returns { cid, size, meta_cid }. |
| GET | /v1/blobs/:cid | Retrieve raw bytes. Supports Range headers. |
| GET | /v1/blobs/:cid/meta | Retrieve metadata envelope. |
| DELETE | /v1/blobs/:cid | Unpin and crypto-shred if encrypted. |
| POST | /v1/upload | Tus resumable upload endpoint. |
| GET | /v1/pins | Pinning Services API v1.0 alias. |
/v1/models · HuggingFace import · Range streaming
| Method | Path | Purpose |
|---|---|---|
| POST | /v1/models | Create a model referencing already-pinned shard CIDs. |
| GET | /v1/models/:name | List all versions of a model. |
| GET | /v1/models/:name/:version | Retrieve version envelope + shard map + model card. |
| POST | /v1/models/:name/versions | Create a new version (immutable, new CID). |
| GET | /v1/models/:name/:version/shards/:shard | Retrieve a shard by logical name. |
| GET | /v1/models/:name/:version/stream | Range-addressable weight streaming. |
| POST | /v1/models/import/huggingface | Import from HuggingFace by repo ID. Async for large models. |
| DELETE | /v1/models/:name/:version | Remove the registry pointer (CID content remains). |
/v1/records · /v1/collections · /v1/views · /v1/schemas
| Method | Path | Purpose |
|---|---|---|
| POST | /v1/records | Write a record. Body: { collection, id?, body, schema_cid? }. |
| GET | /v1/records/:cid | Retrieve any envelope by CID. |
| GET | /v1/records/:collection/:id | Latest version by logical ID. |
| GET | /v1/records/:collection/:id?at=<cid> | Specific historical version by CID. |
| GET | /v1/records/:collection/:id/history | Full version chain (most recent first). |
| DELETE | /v1/records/:collection/:id | Remove collection-head entry. CID remains. |
| POST | /v1/collections/:name/query | Mongo-ish filter query with projections and sorting. |
| POST | /v1/views | Create a named saved query with refresh mode. |
| GET | /v1/views/:name/stream | SSE stream of head CID changes. |
| POST | /v1/schemas | Register a JSON Schema Draft 2020-12 schema. |
The table reflects what each provider actually offers as of April 2026. Only checked entries reflect documented or publicly announced features.
| Capability | NDN IPFS Chain | Pinata | Web3.Storage | Filebase | Fleek |
|---|---|---|---|---|---|
| Blob pinning (PSA v1.0) | ✓ | ✓ | ✓ | ✓ | ✓ |
| AI/ML model registry | ✓ | ✗ | ✗ | ✗ | ✗ |
| Structured records + query | ✓ | ✗ | ✗ | ✗ | ✗ |
| AES-256-GCM encryption (default) | ✓ | ✗ | ✗ | ✗ | ✗ |
| Filecoin cold archival | ✓ | ✗ | ✓ | ✗ | ✗ |
| Crypto-shredding (GDPR erasure) | ✓ | ✗ | ✗ | ✗ | ✗ |
| Resumable upload (Tus) | ✓ | ✗ | ✗ | ✗ | ✗ |
| Open conformance test suite | Planned M1 | ✗ | ✗ | ✗ | ✗ |
| Apache-2.0 reference implementation | ✓ | ✗ | ✗ | ✗ | ✗ |
Three things are now true that were not true three years ago. The protocol-level standard window is open — the ecosystem has not yet fragmented around incompatible structured-data and model-registry approaches.
The Pinning Services API v1.0 standardized blob pinning across providers. The battle for blob interoperability is mostly won. The next layer — models and structured data — has no equivalent standard. The ecosystem is ready for the application-layer protocol stack.
Model weights are deterministic byte sequences. The same model checkpoint, trained identically, produces the same bytes. The missing piece is a standard that makes content addressing first-class at the model-registry layer. Without it, teams keep workunits in centralized silos — HuggingFace, S3 — with no CID.
IPFS's immutability is a feature, not a bug — but it creates a real compliance problem for data that may need erasure. Crypto-shredding (delete the encryption key → all affected CIDs become permanently unreadable) is the correct solution. NDP's envelope encryption makes this a first-class primitive rather than an afterthought.
The API is live, the Structured DB migration is deployed, and the full three-plane API surface is accepting traffic. Health check and dashboard URLs are publicly reachable and return in under 500 ms from us-west1.
ndn-api-1037328355027.us-west1.run.app
ndn-dashboard-1037328355027.us-west1.run.app
Migrated — multi-tenant schema deployed
All three data planes accepting traffic
Published — specs/ndp_protocol.md
github.com/dnkefua/ndn-ipfs-chain
@ndnanalytics/ipfs), Python (ndn-ipfs), and CLI (ndn) SDKs are the primary deliverable of Grant Milestone M1. They are in active development and will publish to npm and PyPI at M1 completion.
Roadmap tied to the IPFS Foundation grant milestones. Total gate payments: $50,000. All targets from grants/MILESTONES.md.
@ndnanalytics/ipfs (JS/TypeScript) v1.0 on npm — ESM + CJS + browser bundles, WebCrypto AES-256-GCM, tus streamingndn-ipfs (Python) v1.0 on PyPI — sync + async (httpx), Pydantic v2 modelsndn CLI — static binary, pin add / list / get / keys rotate, works against any PSA v1.0 backendgateway.ndnipfs.com in us-west, eu-west, ap-southeast — anycast DNS, Cloudflare Workers + R2 fronting Cloud Run<cid>.ipfs.ndnipfs.com with origin isolation?verify=true trustless retrieval — re-hashes content at edge, returns X-Ipfs-Verified headerdocs.ndnipfs.comCalibrated to first-year reality for a solo developer with a free tier. Stated explicitly in the grant application.
Security is layered at the protocol level, not bolted on. Every primitive — tenant isolation, encryption, CID integrity, reserved namespaces — is normative in the NDP v1.0 spec.
Every persistent row and every cache key includes the tenant UUID. Enforcement is layered: application-layer query builder AND Postgres row-level security policies. A bug in application code cannot leak data between tenants without bypassing the SQL layer. Tested as commit b603edf9.
Client-side AES-256-GCM with per-tenant keys managed by KMS. Deleting the envelope key renders all affected CIDs permanently unreadable. Satisfies GDPR Article 17 on immutable storage. Key deletion event is recorded on-chain as tamper-evident proof of destruction.
SHA-256 provides 128-bit collision resistance. JCS canonicalization is deterministic — any deviation from RFC 8785 causes CID divergence. The NDP conformance suite includes canonical-form test vectors that any implementation must pass. Implementations storing >2^64 objects SHOULD migrate to SHA-512.
_schemas, _views, _audit, _meta are privileged namespaces. Direct writes via POST /v1/records are rejected — mutations MUST go through privileged endpoints. This prevents audit log tampering.
Free-tier gateway: 500 req/min/IP, 50 GB/month per anonymous account. CID blocklist integration via Cloudflare and Badbits feeds. Public abuse contact (abuse@ndnipfs.com) and DMCA process in place. Reference implementation's abuse policy published in repository.
The specification is CC-BY 4.0. The reference implementation is Apache-2.0. The commercial product runs on the same open primitives, differentiated by operational features — not by source-code locks.
specs/ndp_protocol.md) — Any party may implement NDP v1.0 without royalties, restrictions, or attribution requirements beyond the license terms.
github.com/dnkefua/ndn-ipfs-chain) — No open-core split. The commercial product runs on the same open primitives.
@ndnanalytics/ndp-conformance and @ndnanalytics/pinning-services-conformance will be published in M1. Any provider can run the suite to claim conformance. Conformance is self-declared against published test vectors — no central certification body required.
Within the grant period, NDP v1.0 will be submitted for IPFS community review via the working-group process, targeting a status equivalent to an IPFS Improvement Proposal. The spec is deliberately storage-backend agnostic — an NDP provider using Arweave instead of Filecoin is a conforming implementation.
Reserved collection names (_schemas, _views, _audit, _meta) and query extensions ($regex, $text, $near) require spec revision — not an implementation decision.