Skip to content

Commit 1d248af

Browse files
NinjaRocksNinjaRocks
committed
Regenerated documentation for increased adoption
1 parent 440a8eb commit 1d248af

File tree

3 files changed

+1265
-4822
lines changed

3 files changed

+1265
-4822
lines changed

‎README.md‎

Lines changed: 210 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,69 +1,231 @@
1-
# <img src="https://github.com/CodeShayk/Schemio/blob/master/Images/data-integration-transparent.png" alt="data" style="width:50px;"/> DataFuse v2.1.0
2-
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/CodeShayk/Schemio/blob/master/LICENSE.md)
3-
[![Master-Build](https://github.com/CodeShayk/Schemio/actions/workflows/Build-Master.yml/badge.svg)](https://github.com/CodeShayk/Schemio/actions/workflows/Build-Master.yml)
4-
[![GitHub Release](https://img.shields.io/github/v/release/CodeShayk/Schemio?logo=github&sort=semver)](https://github.com/CodeShayk/Schemio/releases/latest)
5-
[![Master-CodeQL](https://github.com/CodeShayk/Schemio/actions/workflows/Master-CodeQL.yml/badge.svg)](https://github.com/CodeShayk/Schemio/actions/workflows/Master-CodeQL.yml)
1+
# <img src="https://github.com/CodeShayk/DataFuse/blob/master/Images/data-integration-transparent.png" alt="data" style="width:50px;"/> DataFuse
2+
3+
### Like GraphQL, but for your heterogeneous backend systems
4+
5+
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://github.com/CodeShayk/DataFuse/blob/master/LICENSE.md)
6+
[![Master-Build](https://github.com/CodeShayk/DataFuse/actions/workflows/Build-Master.yml/badge.svg)](https://github.com/CodeShayk/DataFuse/actions/workflows/Build-Master.yml)
7+
[![GitHub Release](https://img.shields.io/github/v/release/CodeShayk/DataFuse?logo=github&sort=semver)](https://github.com/CodeShayk/DataFuse/releases/latest)
8+
[![Master-CodeQL](https://github.com/CodeShayk/DataFuse/actions/workflows/Master-CodeQL.yml/badge.svg)](https://github.com/CodeShayk/DataFuse/actions/workflows/Master-CodeQL.yml)
69
[![.Net 9.0](https://img.shields.io/badge/.Net-9.0-blue)](https://dotnet.microsoft.com/en-us/download/dotnet/9.0)
7-
--
8-
DataFuse is a powerful .NET library designed to aggregate data from heterogeneous data stores using a schema-driven approach. It enables developers to hydrate complex object graphs by fetching data from multiple sources (SQL databases, Web APIs, NoSQL stores) using XPath and JSONPath schema mappings.
9-
#### Nuget Packages
10-
| Package | Latest | Details |
11-
| --------| --------| --------|
12-
| DataFuse.Integration|[![NuGet version](https://badge.fury.io/nu/DataFuse.Integration.svg)](https://badge.fury.io/nu/DataFuse.Integration) | Provides `core` functionality to configure nested queries and transformers. With ability to map schema paths (XPath/JSONPath) to entity's object graph. `No QueryEngine` provided and requires implementing IQueryEngine to execute IQuery instances. |
13-
| DataFuse.Adapters.SQL|[![NuGet version](https://badge.fury.io/nu/DataFuse.Adapters.SQL.svg)](https://badge.fury.io/nu/DataFuse.Adapters.SQL)| Provides DataFuse with query engine using `Dapper` to execute SQL queries. |
14-
| DataFuse.Adapters.EntityFramework|[![NuGet version](https://badge.fury.io/nu/DataFuse.Adapters.EntityFramework.svg)](https://badge.fury.io/nu/DataFuse.Adapters.EntityFramework)| Provides DataFuse with `Entity Framework` query engine to execute queries using DbContext. |
15-
| DataFuse.Adapters.WebAPI|[![NuGet version](https://badge.fury.io/nu/DataFuse.Adapters.WebAPI.svg)](https://badge.fury.io/nu/DataFuse.Adapters.WebAPI)| Provides DataFuse with `Web Api` query engine to execute apis using HttpClient. |
16-
17-
## Concept
18-
### What is DataFuse?
19-
`DataFuse` is a data aggregation framework using queries that can target different data platforms.
20-
21-
Key benefits:
22-
- allows fetching `aggregated` data from `heterogeneous` data storages. You could combine queries targetting different data platforms (example. `SQL`, `API`, `Cache`) to return an aggregated data `entity`.
23-
- allows `conditional` fetching of `parts` of the aggregated data entity. You could retrieve parts of object graph in the aggregated entity by specifying schema paths (using `XPath` or `JSonPath`) to identify respective sections.
24-
25-
## Getting Started?
26-
### i. Installation
27-
Install the latest nuget package as appropriate for `Core`, `Web API`, `SQL` using `Dapper` or `EntityFramework` using commands below.
28-
29-
`DataFuse.Integration` - for installing DataFuse for `bespoke` implementation of query engine.
10+
11+
**DataFuse** is a declarative .NET framework that aggregates data from SQL databases, REST APIs, and Entity Framework into unified, strongly-typed objects — replacing hundreds of lines of manual orchestration code with clean, schema-driven configuration.
12+
13+
---
14+
15+
## The Problem
16+
17+
Building a product page? You need inventory from SQL Server, pricing from a REST API, and reviews from an external service. That's 50+ lines of manual fetch-assemble code — sequential, tightly coupled, and repeated across your codebase.
18+
19+
```csharp
20+
// Without DataFuse: manual orchestration everywhere
21+
var product = await db.QueryFirstAsync<ProductRecord>("SELECT * FROM Products WHERE Id = @Id", new { Id = productId });
22+
var pricing = await httpClient.GetFromJsonAsync<PricingResponse>($"https://pricing-api/products/{productId}");
23+
var reviews = await httpClient.GetFromJsonAsync<ReviewResponse[]>($"https://reviews-api/products/{productId}");
24+
25+
product.Price = pricing.Price;
26+
product.Reviews = reviews.Select(r => new Review { Rating = r.Stars, Comment = r.Text }).ToArray();
27+
```
28+
29+
## The Solution
30+
31+
```csharp
32+
// With DataFuse: declare once, use everywhere
33+
var product = dataProvider.GetData(new ProductRequest { ProductId = 42 });
3034
```
31-
NuGet\Install-Package DataFuse.Integration
35+
36+
DataFuse automatically executes the SQL query first, then runs pricing and reviews API calls **in parallel**, passes parent results to child queries, and assembles the final typed object.
37+
38+
---
39+
40+
## Quick Start
41+
42+
### 1. Install
43+
44+
```bash
45+
dotnet add package DataFuse.Integration
46+
dotnet add package DataFuse.Adapters.SQL # SQL with Dapper
47+
dotnet add package DataFuse.Adapters.WebAPI # REST APIs
3248
```
33-
`DataFuse.Adapters.SQL` - for installing DataFuse for SQL with `Dapper` engine.
49+
50+
### 2. Define Your Entity
51+
52+
```csharp
53+
public class Product : IEntity
54+
{
55+
public int ProductId { get; set; }
56+
public string Name { get; set; }
57+
public decimal Price { get; set; }
58+
public Review[] Reviews { get; set; }
59+
}
3460
```
35-
NuGet\Install-Package DataFuse.Adapters.SQL
61+
62+
### 3. Create Queries
63+
64+
```csharp
65+
// SQL query — fetches product from database
66+
public class ProductQuery : SQLQuery<ProductResult>
67+
{
68+
protected override Func<IDbConnection, Task<ProductResult>> GetQuery(
69+
IDataContext context, IQueryResult parentQueryResult)
70+
{
71+
var request = (ProductRequest)context.Request;
72+
return connection => connection.QueryFirstOrDefaultAsync<ProductResult>(
73+
"SELECT ProductId as Id, Name, Price FROM Products WHERE ProductId = @Id",
74+
new { Id = request.ProductId });
75+
}
76+
}
77+
78+
// API query — fetches reviews from external service, receives parent result
79+
public class ReviewsApiQuery : WebQuery<CollectionResult<ReviewResult>>
80+
{
81+
public ReviewsApiQuery() : base("https://api.reviews.com/") { }
82+
83+
protected override Func<Uri> GetQuery(IDataContext context, IQueryResult parentQueryResult)
84+
{
85+
var product = (ProductResult)parentQueryResult;
86+
return () => new Uri($"products/{product.Id}/reviews", UriKind.Relative);
87+
}
88+
}
3689
```
37-
`DataFuse.Adapters.EntityFramework` - for installing DataFuse for SQL with `EntityFramework` engine.
90+
91+
### 4. Create Transformers
92+
93+
```csharp
94+
public class ProductTransform : BaseTransformer<ProductResult, Product>
95+
{
96+
public override void Transform(ProductResult queryResult, Product entity)
97+
{
98+
entity.ProductId = queryResult.Id;
99+
entity.Name = queryResult.Name;
100+
entity.Price = queryResult.Price;
101+
}
102+
}
103+
104+
public class ReviewsTransform : BaseTransformer<CollectionResult<ReviewResult>, Product>
105+
{
106+
public override void Transform(CollectionResult<ReviewResult> queryResult, Product entity)
107+
{
108+
entity.Reviews = queryResult?.Select(r => new Review
109+
{
110+
ReviewId = r.Id, Comment = r.Comment, Rating = r.Rating
111+
}).ToArray() ?? Array.Empty<Review>();
112+
}
113+
}
38114
```
39-
NuGet\Install-Package DataFuse.Adapters.EntityFramework
115+
116+
### 5. Configure the Schema
117+
118+
```csharp
119+
public class ProductConfiguration : EntityConfiguration<Product>
120+
{
121+
public override IEnumerable<Mapping<Product, IQueryResult>> GetSchema()
122+
{
123+
return CreateSchema.For<Product>()
124+
.Map<ProductQuery, ProductTransform>(For.Paths("product"),
125+
product => product.Dependents
126+
.Map<ReviewsApiQuery, ReviewsTransform>(For.Paths("product/reviews")))
127+
.End();
128+
}
129+
}
40130
```
41-
`DataFuse.Adapters.WebAPI` - for installing DataFuse for Web API with `HttpClient` engine.
131+
132+
### 6. Register & Use
133+
134+
```csharp
135+
// DI registration
136+
services.UseDataFuse()
137+
.WithEngine(c => new QueryEngine(sqlConfiguration))
138+
.WithEngine<DataFuse.Adapters.WebAPI.QueryEngine>()
139+
.WithPathMatcher(c => new XPathMatcher())
140+
.WithEntityConfiguration<Product>(c => new ProductConfiguration());
141+
142+
services.AddHttpClient();
42143
```
43-
NuGet\Install-Package DataFuse.Adapters.WebAPI
144+
145+
```csharp
146+
// Usage — one line to get fully hydrated data
147+
public class ProductService
148+
{
149+
private readonly IDataProvider<Product> _dataProvider;
150+
151+
public ProductService(IDataProvider<Product> dataProvider) => _dataProvider = dataProvider;
152+
153+
public Product GetProduct(int productId)
154+
=> _dataProvider.GetData(new ProductRequest { ProductId = productId });
155+
156+
// Selective loading — only fetch product + reviews, skip other paths
157+
public Product GetProductWithReviews(int productId)
158+
=> _dataProvider.GetData(new ProductRequest
159+
{
160+
ProductId = productId,
161+
SchemaPaths = new[] { "product", "product/reviews" }
162+
});
163+
}
44164
```
45-
### ii. Developer Guide
46165

47-
Please see [Developer Guide](https://github.com/CodeShayk/Schemio/wiki) for complete details to use DataFuse in your project.
166+
---
167+
168+
## Key Features
169+
170+
| Feature | Description |
171+
|---|---|
172+
| **Declarative Schema** | Define data relationships once with fluent configuration |
173+
| **Automatic Parallelism** | Sibling queries run in parallel — no `Task.WhenAll` boilerplate |
174+
| **Selective Loading** | Fetch only the data paths the consumer needs via `SchemaPaths` |
175+
| **Parent-Child Dependencies** | Parent results flow to child queries automatically |
176+
| **Cross-Source Mixing** | Combine SQL + REST API + EF Core queries in one entity |
177+
| **Type Safety** | Strongly-typed queries, results, and transformers |
178+
| **Built-in Caching** | `[CacheResult]` attribute for expensive query results |
179+
| **Custom Adapters** | Add any data source by implementing `IQueryEngine` |
180+
181+
---
182+
183+
## Packages
184+
185+
| Package | Purpose | Install |
186+
|---|---|---|
187+
| **DataFuse.Integration** | Core orchestration, DI, helpers | `dotnet add package DataFuse.Integration` |
188+
| **DataFuse.Adapters.SQL** | SQL via Dapper | `dotnet add package DataFuse.Adapters.SQL` |
189+
| **DataFuse.Adapters.EntityFramework** | EF Core | `dotnet add package DataFuse.Adapters.EntityFramework` |
190+
| **DataFuse.Adapters.WebAPI** | REST APIs via HttpClient | `dotnet add package DataFuse.Adapters.WebAPI` |
191+
| **DataFuse.Adapters.Abstraction** | Interfaces & base classes | Included as dependency |
192+
193+
### Compatibility
194+
195+
| Package | .NET | .NET Standard | .NET Framework |
196+
|---|---|---|---|
197+
| DataFuse.Integration | 9.0+ | 2.0, 2.1 | 4.6.2+ |
198+
| DataFuse.Adapters.SQL | 9.0+ | 2.1 | 4.6.2+ |
199+
| DataFuse.Adapters.EntityFramework | 9.0+ | - | - |
200+
| DataFuse.Adapters.WebAPI | 9.0+ | 2.0, 2.1 | 4.6.2+ |
201+
202+
---
203+
204+
## Documentation
205+
206+
See the [Complete Developer Guide](https://codeshayk.github.io/DataFuse/) for detailed documentation including:
207+
- Real-world use cases (e-commerce, customer 360, reporting)
208+
- Core concepts deep dive
209+
- Query implementation guides (SQL, EF Core, Web API)
210+
- Transformer patterns
211+
- Advanced features (caching, selective loading, custom engines)
212+
- Architecture overview
213+
214+
---
48215

49216
## Support
50217

51-
If you are having problems, please let me know by [raising a new issue](https://github.com/CodeShayk/Schemio/issues/new/choose).
218+
If you are having problems, please let us know by [raising a new issue](https://github.com/CodeShayk/DataFuse/issues/new/choose).
52219

53220
## License
54221

55222
This project is licensed with the [MIT license](LICENSE).
56223

57224
## Version History
58-
The main branch is now on .NET 9.0. The following previous versions are available:
59-
| Version | Release Notes | Developer Guide |
60-
| -------- | --------|--------|
61-
| [`v2.0.0`](https://github.com/CodeShayk/Schemio/tree/v2.0.0) | [Notes](https://github.com/CodeShayk/Schemio/releases/tag/v2.0.0) | [Guide](https://github.com/CodeShayk/Schemio/blob/v2.0.0/index.md) |
62-
| [`v1.0.0`](https://github.com/CodeShayk/Schemio/tree/v1.0.0) | [Notes](https://github.com/CodeShayk/Schemio/releases/tag/v1.0.0) | [Guide](https://github.com/CodeShayk/Schemio/blob/v1.0.0/index.md) |
63-
64-
## Credits
65-
Thank you for reading. Please fork, explore, contribute and report. Happy Coding !! :)
66-
67-
68225

226+
The main branch is now on .NET 9.0. Previous versions:
69227

228+
| Version | Release Notes | Developer Guide |
229+
|---|---|---|
230+
| [`v2.0.0`](https://github.com/CodeShayk/DataFuse/tree/v2.0.0) | [Notes](https://github.com/CodeShayk/DataFuse/releases/tag/v2.0.0) | [Guide](https://github.com/CodeShayk/DataFuse/blob/v2.0.0/index.md) |
231+
| [`v1.0.0`](https://github.com/CodeShayk/DataFuse/tree/v1.0.0) | [Notes](https://github.com/CodeShayk/DataFuse/releases/tag/v1.0.0) | [Guide](https://github.com/CodeShayk/DataFuse/blob/v1.0.0/index.md) |

0 commit comments

Comments
 (0)