Skip to content

Conversation

@sanjeev23oct
Copy link

Postgres Function support added

Why make this change?

  • Reference associated issue using # syntax. e.g. Closes #XX
    • Include summary (1-2 sentences) of linked issue to avoid redirecting reviewers to different pages.
    • Include links to any additional discussion threads related to this change.

What is this change?

  • Summary of how your changes work to give reviewers context of your intent.
    • Links to relevant documentation / StackOverflow, if applicable.

How was this tested?

  • Integration Tests
  • Unit Tests

Sample Request(s)

  • Example REST and/or GraphQL request to demonstrate modifications
  • Example of CLI usage to demonstrate modifications

Postgres Function support added
@sanjeev23oct
Copy link
Author

@microsoft-github-policy-service agree

@seantleonard
Copy link
Contributor

Thank you for your contribution , @sanjeev23oct. Can you fill out the PR description template with contextual info? Please also add tests for validating your changes, which will help us review.

/// <summary>
/// Takes a string version of a MySql data type and returns its .NET common language runtime (CLR) counterpart
/// TODO: For MySql stored procedure support, this needs to be implemented.
/// Takes a string version of a PostgreSql data type and returns its .NET common language runtime (CLR) counterpart
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If there are changes related to MySQL as well, can you move them in a separate PR?

Since, the PR title says it's for Postgres

@vs-dsva
Copy link

vs-dsva commented Aug 18, 2025

What is the status of this?
Can it be merged?

@JerryNixon
Copy link
Contributor

Resurrecting this with the PG team for review.

@guushoekman
Copy link

Any progress on this? @JerryNixon do you know anything by any chance?

@JerryNixon
Copy link
Contributor

Any progress on this? @JerryNixon do you know anything by any chance?

Nothing yet. If there is a timeline, and there is not, it is a longer timeline.

@JerryNixon JerryNixon removed the request for review from ravishetye December 16, 2025 20:37
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds PostgreSQL function support to the Data API Builder, enabling the system to handle PostgreSQL database functions in addition to stored procedures. The implementation includes metadata providers, query builders, and configuration models to support this new entity source type.

Key Changes:

  • Added Function as a new EntitySourceType alongside Table, View, and StoredProcedure
  • Implemented PostgreSQL-specific function metadata retrieval and query building
  • Added SQL-to-CLR type mapping for PostgreSQL (and incorrectly for MySQL)

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 18 comments.

Show a summary per file
File Description
src/Config/ObjectModel/EntitySourceType.cs Adds Function enum value to support function entities
src/Config/DatabasePrimitives/DatabaseObject.cs Extends SourceDefinition property to handle Function source type
src/Core/Services/MetadataProviders/SqlMetadataProvider.cs Adds FillSchemaForFunctionAsync method to retrieve function metadata from PostgreSQL information schema and updates entity processing logic
src/Core/Services/MetadataProviders/PostgreSqlMetadataProvider.cs Implements SqlToCLRType for PostgreSQL type conversions and overrides stored procedure schema filling
src/Core/Services/MetadataProviders/MySqlMetadataProvider.cs Incorrectly implements PostgreSQL type mappings instead of MySQL-specific mappings
src/Core/Resolvers/PostgresQueryBuilder.cs Implements function execution query building with parameter formatting
src/Core/Resolvers/BaseQueryStructure.cs Modifies parameter handling to use named parameters directly
Comments suppressed due to low confidence (1)

src/Core/Resolvers/PostgresQueryBuilder.cs:133

  • Condition is always false because of ... == ....
                if (value == null)

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +132 to +133
/// Takes a string version of a PostgreSql data type and returns its .NET common language runtime (CLR) counterpart
/// TODO: For PostgreSql stored procedure support, this needs to be implemented.
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment incorrectly refers to "PostgreSql" instead of "MySql". This method implements type mapping for MySql, not PostgreSql.

Suggested change
/// Takes a string version of a PostgreSql data type and returns its .NET common language runtime (CLR) counterpart
/// TODO: For PostgreSql stored procedure support, this needs to be implemented.
/// Takes a string version of a MySql data type and returns its .NET common language runtime (CLR) counterpart
/// TODO: For MySql stored procedure support, this needs to be implemented.

Copilot uses AI. Check for mistakes.
Comment on lines +139 to +245
case "boolean":
case "bool":
return typeof(bool);

case "smallint":
return typeof(short);

case "integer":
case "int":
return typeof(int);

case "bigint":
return typeof(long);

case "real":
return typeof(float);

case "double precision":
return typeof(double);

case "numeric":
case "decimal":
return typeof(decimal);

case "money":
return typeof(decimal);

case "character varying":
case "varchar":
case "character":
case "char":
case "text":
return typeof(string);

case "bytea":
return typeof(byte[]);

case "date":
return typeof(DateTime);

case "timestamp":
case "timestamp without time zone":
return typeof(DateTime);

case "timestamp with time zone":
return typeof(DateTimeOffset);

case "time":
case "time without time zone":
return typeof(TimeSpan);

case "time with time zone":
return typeof(DateTimeOffset);

case "interval":
return typeof(TimeSpan);

case "uuid":
return typeof(Guid);

case "json":
case "jsonb":
return typeof(string);

case "xml":
return typeof(string);

case "inet":
return typeof(System.Net.IPAddress);

case "cidr":
return typeof(ValueTuple<System.Net.IPAddress, int>);

case "macaddr":
return typeof(System.Net.NetworkInformation.PhysicalAddress);

case "bit":
case "bit varying":
return typeof(BitArray);

case "point":
return typeof((double, double));

case "line":
return typeof(string); // Implement a custom type if needed

case "lseg":
return typeof((double, double)[]);

case "box":
return typeof((double, double)[]);

case "path":
return typeof((double, double)[]);

case "polygon":
return typeof((double, double)[]);

case "circle":
return typeof((double, double, double));

case "tsvector":
return typeof(string); // Implement a custom type if needed

case "tsquery":
return typeof(string); // Implement a custom type if needed

Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The SqlToCLRType implementation in MySqlMetadataProvider contains PostgreSQL-specific type mappings (like "boolean", "double precision", "bytea", "jsonb", "inet", "cidr", "macaddr", "bit varying", "tsvector", "tsquery") rather than MySQL type mappings. MySQL uses different type names such as "tinyint", "mediumint", "blob", "longtext", "datetime", etc. This implementation appears to have been incorrectly copied from the PostgreSQL provider.

Suggested change
case "boolean":
case "bool":
return typeof(bool);
case "smallint":
return typeof(short);
case "integer":
case "int":
return typeof(int);
case "bigint":
return typeof(long);
case "real":
return typeof(float);
case "double precision":
return typeof(double);
case "numeric":
case "decimal":
return typeof(decimal);
case "money":
return typeof(decimal);
case "character varying":
case "varchar":
case "character":
case "char":
case "text":
return typeof(string);
case "bytea":
return typeof(byte[]);
case "date":
return typeof(DateTime);
case "timestamp":
case "timestamp without time zone":
return typeof(DateTime);
case "timestamp with time zone":
return typeof(DateTimeOffset);
case "time":
case "time without time zone":
return typeof(TimeSpan);
case "time with time zone":
return typeof(DateTimeOffset);
case "interval":
return typeof(TimeSpan);
case "uuid":
return typeof(Guid);
case "json":
case "jsonb":
return typeof(string);
case "xml":
return typeof(string);
case "inet":
return typeof(System.Net.IPAddress);
case "cidr":
return typeof(ValueTuple<System.Net.IPAddress, int>);
case "macaddr":
return typeof(System.Net.NetworkInformation.PhysicalAddress);
case "bit":
case "bit varying":
return typeof(BitArray);
case "point":
return typeof((double, double));
case "line":
return typeof(string); // Implement a custom type if needed
case "lseg":
return typeof((double, double)[]);
case "box":
return typeof((double, double)[]);
case "path":
return typeof((double, double)[]);
case "polygon":
return typeof((double, double)[]);
case "circle":
return typeof((double, double, double));
case "tsvector":
return typeof(string); // Implement a custom type if needed
case "tsquery":
return typeof(string); // Implement a custom type if needed
case "tinyint":
return typeof(sbyte); // MySQL tinyint is signed by default
case "tinyint unsigned":
return typeof(byte);
case "bool":
case "boolean":
return typeof(bool);
case "smallint":
return typeof(short);
case "smallint unsigned":
return typeof(ushort);
case "mediumint":
return typeof(int);
case "mediumint unsigned":
return typeof(uint);
case "int":
case "integer":
return typeof(int);
case "int unsigned":
case "integer unsigned":
return typeof(uint);
case "bigint":
return typeof(long);
case "bigint unsigned":
return typeof(ulong);
case "float":
return typeof(float);
case "double":
case "double precision":
return typeof(double);
case "decimal":
case "dec":
case "fixed":
case "numeric":
return typeof(decimal);
case "bit":
return typeof(ulong); // MySQL BIT can be up to 64 bits
case "char":
case "nchar":
case "varchar":
case "nvarchar":
case "text":
case "tinytext":
case "mediumtext":
case "longtext":
return typeof(string);
case "binary":
case "varbinary":
case "blob":
case "tinyblob":
case "mediumblob":
case "longblob":
return typeof(byte[]);
case "enum":
case "set":
return typeof(string);
case "date":
case "datetime":
case "timestamp":
return typeof(DateTime);
case "time":
return typeof(TimeSpan);
case "year":
return typeof(int);
case "json":
return typeof(string);

Copilot uses AI. Check for mistakes.
information_schema.routines r
ON p.specific_name = r.specific_name
WHERE
r.routine_schema = 'public'
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The routine_schema is hardcoded to 'public' instead of using a parameter. This will cause the query to fail for functions in non-public schemas. The query should use a parameterized value like the function name.

Copilot uses AI. Check for mistakes.
WHERE
r.routine_schema = 'public'
and p.parameter_mode = 'OUT'
AND r.routine_name = '{procedureName}'
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The procedureName parameter is directly concatenated into the SQL query string, making this code vulnerable to SQL injection attacks. The parameter should be properly escaped or parameterized.

Copilot uses AI. Check for mistakes.
}

// Handle numeric and other types
return value == null ? string.Empty : value.ToString();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The FormatParameterValue method directly embeds parameter values into the SQL query string, creating a SQL injection vulnerability. Named parameters should use proper parameterization instead of inline value formatting.

Suggested change
return value == null ? string.Empty : value.ToString();
throw new NotSupportedException("Direct formatting of parameter values is not supported. Use parameterized queries instead.");

Copilot uses AI. Check for mistakes.
View,
[EnumMember(Value = "stored-procedure")] StoredProcedure
[EnumMember(Value = "stored-procedure")] StoredProcedure,
Function
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Function entity source type is missing the EnumMember attribute with a serialization value. For consistency with StoredProcedure which has [EnumMember(Value = "stored-procedure")], this should have an attribute like [EnumMember(Value = "function")].

Suggested change
Function
[EnumMember(Value = "function")] Function

Copilot uses AI. Check for mistakes.

using (NpgsqlDataReader reader = await procedureCommand.ExecuteReaderAsync())
{
procedureMetadata = new DataTable();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disposable 'DataTable' is created but not disposed.

Copilot uses AI. Check for mistakes.

using (NpgsqlDataReader reader = await parameterCommand.ExecuteReaderAsync())
{
parameterMetadata = new DataTable();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disposable 'DataTable' is created but not disposed.

Copilot uses AI. Check for mistakes.
}

// Handle numeric and other types
return value == null ? string.Empty : value.ToString();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Condition is always false because of ... == ....

Suggested change
return value == null ? string.Empty : value.ToString();
return value.ToString();

Copilot uses AI. Check for mistakes.
EntitySourceType.Table => ((DatabaseTable)this).TableDefinition,
EntitySourceType.View => ((DatabaseView)this).ViewDefinition,
EntitySourceType.StoredProcedure => ((DatabaseStoredProcedure)this).StoredProcedureDefinition,
EntitySourceType.Function => ((DatabaseStoredProcedure)this).StoredProcedureDefinition,
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Downcasting 'this' from DatabaseObject to DatabaseStoredProcedure introduces a dependency cycle between the two types.

Copilot uses AI. Check for mistakes.
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 15 comments.

Comments suppressed due to low confidence (2)

src/Core/Resolvers/PostgresQueryBuilder.cs:142

  • Condition is always false because of ... == ....
                if (value == null)

src/Core/Resolvers/PostgresQueryBuilder.cs:165

  • Condition is always false because of ... == ....
            return value == null ? string.Empty : value.ToString();

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

}

// Handle numeric and other types
return value == null ? string.Empty : value.ToString();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This ternary expression on line 165 is redundant. If value is null, the method already returns "NULL" at line 137. The null check here is unnecessary.

Suggested change
return value == null ? string.Empty : value.ToString();
return value.ToString();

Copilot uses AI. Check for mistakes.
information_schema.routines r
ON p.specific_name = r.specific_name
WHERE
r.routine_schema = 'public'
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The schema name is hardcoded to 'public' in this query. This should use a parameter or variable to support schemas other than the default 'public' schema, similar to how it's done elsewhere in the codebase.

Copilot uses AI. Check for mistakes.
Comment on lines +142 to +149
if (value == null)
{
value = string.Empty;
}
// Handle string values, escaping single quotes
#pragma warning disable CS8602 // Dereference of a possibly null reference.
return $"{value.ToString().Replace("'", "''")}";
#pragma warning restore CS8602 // Dereference of a possibly null reference.
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The suppression of CS8602 (Dereference of a possibly null reference) is unnecessary because the null check on line 142-145 already handles the null case (though that code is unreachable due to the earlier null check on line 135-138). After removing the unreachable code, this pragma can also be removed since value.ToString() will not be null at this point.

Suggested change
if (value == null)
{
value = string.Empty;
}
// Handle string values, escaping single quotes
#pragma warning disable CS8602 // Dereference of a possibly null reference.
return $"{value.ToString().Replace("'", "''")}";
#pragma warning restore CS8602 // Dereference of a possibly null reference.
// Handle string values, escaping single quotes
return $"{value.ToString().Replace("'", "''")}";

Copilot uses AI. Check for mistakes.
if (!string.IsNullOrEmpty(paramName))
{
Parameters.Add(encodedParamName,
Parameters.Add(paramName,
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change from encodedParamName to paramName in the Parameters dictionary breaks the existing parameter encoding mechanism. This could cause parameter name collisions if the same parameter name is used multiple times, since the encoded names were designed to be unique. This change affects all database query structures, not just PostgreSQL functions.

Suggested change
Parameters.Add(paramName,
Parameters.Add(encodedParamName,

Copilot uses AI. Check for mistakes.
// of the stored procedure.
string queryForResultSetDetails = SqlQueryBuilder.BuildStoredProcedureResultDetailsQuery(
dbStoredProcedureName);
storedProcedureName);
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The parameter passed to BuildStoredProcedureResultDetailsQuery was changed from dbStoredProcedureName (which includes the schema prefix as seen on line 1354: schemaName.storedProcedureName) to just storedProcedureName (the bare procedure name). This changes the behavior of the query being built and may cause issues if the query builder expects the fully qualified name.

Suggested change
storedProcedureName);
dbStoredProcedureName);

Copilot uses AI. Check for mistakes.
List<string> parameterList = new();
foreach (KeyValuePair<string, object> param in parameters)
{
parameterList.Add($"{QuoteIdentifier(param.Key)} := {FormatParameterValue(param.Value)}");
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The named parameter syntax used here (parameter_name := value) is PostgreSQL-specific. However, this implementation doesn't use parameterized queries, which means the values are being embedded directly into the SQL string. This approach is vulnerable to SQL injection attacks. The implementation should use proper parameterized queries instead of string formatting the values directly into the SQL.

Copilot uses AI. Check for mistakes.
Comment on lines +525 to +578
protected virtual async Task FillSchemaForFunctionAsync(
Entity procedureEntity,
string entityName,
string schemaName,
string storedProcedureSourceName,
StoredProcedureDefinition storedProcedureDefinition)
{
using NpgsqlConnection conn = new(ConnectionString);
DataTable procedureMetadata;
DataTable parameterMetadata;

try
{
await QueryExecutor.SetManagedIdentityAccessTokenIfAnyAsync(conn, _dataSourceName);
await conn.OpenAsync();

// Retrieve procedure metadata from PostgreSQL information schema
string procedureMetadataQuery = @"
SELECT routine_name
FROM information_schema.routines
WHERE routine_schema = @schemaName AND routine_name = @procedureName
";

using (NpgsqlCommand procedureCommand = new(procedureMetadataQuery, conn))
{
procedureCommand.Parameters.AddWithValue("@schemaName", schemaName);
procedureCommand.Parameters.AddWithValue("@procedureName", storedProcedureSourceName);

using (NpgsqlDataReader reader = await procedureCommand.ExecuteReaderAsync())
{
procedureMetadata = new DataTable();
procedureMetadata.Load(reader);
}
}

// Throw exception if procedure does not exist
if (procedureMetadata.Rows.Count == 0)
{
throw new DataApiBuilderException(
message: $"No stored procedure definition found for the given database object {storedProcedureSourceName}",
statusCode: HttpStatusCode.ServiceUnavailable,
subStatusCode: DataApiBuilderException.SubStatusCodes.ErrorInInitialization);
}

// Retrieve parameter metadata from PostgreSQL information schema
string parameterMetadataQuery = @"
SELECT
p.parameter_name,
p.data_type
FROM
information_schema.parameters p
JOIN
information_schema.routines r ON p.specific_name = r.specific_name
WHERE
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using NpgsqlConnection directly in SqlMetadataProvider couples the base provider class to PostgreSQL-specific implementation. This breaks the abstraction since SqlMetadataProvider is the base class for multiple database types. The PostgreSQL-specific function handling should be in PostgreSqlMetadataProvider, not the base class.

Suggested change
protected virtual async Task FillSchemaForFunctionAsync(
Entity procedureEntity,
string entityName,
string schemaName,
string storedProcedureSourceName,
StoredProcedureDefinition storedProcedureDefinition)
{
using NpgsqlConnection conn = new(ConnectionString);
DataTable procedureMetadata;
DataTable parameterMetadata;
try
{
await QueryExecutor.SetManagedIdentityAccessTokenIfAnyAsync(conn, _dataSourceName);
await conn.OpenAsync();
// Retrieve procedure metadata from PostgreSQL information schema
string procedureMetadataQuery = @"
SELECT routine_name
FROM information_schema.routines
WHERE routine_schema = @schemaName AND routine_name = @procedureName
";
using (NpgsqlCommand procedureCommand = new(procedureMetadataQuery, conn))
{
procedureCommand.Parameters.AddWithValue("@schemaName", schemaName);
procedureCommand.Parameters.AddWithValue("@procedureName", storedProcedureSourceName);
using (NpgsqlDataReader reader = await procedureCommand.ExecuteReaderAsync())
{
procedureMetadata = new DataTable();
procedureMetadata.Load(reader);
}
}
// Throw exception if procedure does not exist
if (procedureMetadata.Rows.Count == 0)
{
throw new DataApiBuilderException(
message: $"No stored procedure definition found for the given database object {storedProcedureSourceName}",
statusCode: HttpStatusCode.ServiceUnavailable,
subStatusCode: DataApiBuilderException.SubStatusCodes.ErrorInInitialization);
}
// Retrieve parameter metadata from PostgreSQL information schema
string parameterMetadataQuery = @"
SELECT
p.parameter_name,
p.data_type
FROM
information_schema.parameters p
JOIN
information_schema.routines r ON p.specific_name = r.specific_name
WHERE
protected virtual Task FillSchemaForFunctionAsync(
Entity procedureEntity,
string entityName,
string schemaName,
string storedProcedureSourceName,
StoredProcedureDefinition storedProcedureDefinition)
{
throw new NotImplementedException("FillSchemaForFunctionAsync must be implemented in a database-specific provider.");
}

Copilot uses AI. Check for mistakes.
Comment on lines +142 to +145
if (value == null)
{
value = string.Empty;
}
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is unreachable code here. The null check on line 135 already returns "NULL" if value is null, so lines 142-145 will never execute because value cannot be null at that point. This check should be removed.

Suggested change
if (value == null)
{
value = string.Empty;
}

Copilot uses AI. Check for mistakes.

using (NpgsqlDataReader reader = await procedureCommand.ExecuteReaderAsync())
{
procedureMetadata = new DataTable();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disposable 'DataTable' is created but not disposed.

Copilot uses AI. Check for mistakes.

using (NpgsqlDataReader reader = await parameterCommand.ExecuteReaderAsync())
{
parameterMetadata = new DataTable();
Copy link

Copilot AI Dec 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Disposable 'DataTable' is created but not disposed.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants