Skip to content

refactor!: expanding return value of getLogsByTags #19233

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
benesjan wants to merge 7 commits into
base: 12-18-feat_integrating_new_log_sync
Choose a base branch
from
+237 −399
Open

refactor!: expanding return value of getLogsByTags #19233

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
b1c756a
feat: expanding return value of getLogsByTags
benesjan Dec 23, 2025
56ace7f
dropping unused values from the type
benesjan Dec 23, 2025
64e0cfb
cleanup
benesjan Dec 23, 2025
468d96d
cleanup
benesjan Dec 23, 2025
ee96435
test fixes
benesjan Dec 24, 2025
6b89008
migration notes
benesjan Dec 24, 2025
eb9692a
final cleanup
benesjan Dec 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 24 additions & 16 deletions docs/docs-developers/docs/resources/migration_notes.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,26 +9,34 @@ Aztec is in full-speed development. Literally every version breaks compatibility

## TBD

### [Aztec Node] changes to `getLogsByTags` endpoint

`getLogsByTags` endpoint has been optimized for our new log sync algorithm and these are the changes:

- The `logsPerTag` pagination argument has been removed. Pagination was unnecessary here, since multiple logs per tag typically only occur if several devices are sending logs from the same sender to a recipient, which is unlikely to generate enough logs to require pagination.
- The structure of `TxScopedL2Log` has been revised to meet the requirements of our new log sync algorithm.
- The endpoint has been separated into two versions: `getPrivateLogsByTags` and `getPublicLogsByTagsFromContract`. This change was made because it was never desirable in PXE to mix public and private logs. The public version requires both a `Tag` and a contract address as input. In contrast to the private version—which uses `SiloedTag` (a tag that hashes the raw tag with the emitting contract's address)—the public version uses the raw `Tag` type, since kernels do not hash the tag with the contract address for public logs.

### [AVM] Gas cost multipliers for public execution to reach simulation/proving parity

Gas costs for several AVM opcodes have been adjusted with multipliers to better align public simulation costs with actual proving costs.

Copy link
Contributor Author

@benesjan benesjan Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just run formatting below.

| Opcode | Multiplier | Previous Cost | New Cost |
|--------|------------|---------------|----------|
| FDIV | 25x | 9 | 225 |
| SLOAD | 10x | 129 | 1,290 |
| SSTORE | 20x | 1,657 | 33,140 |
| NOTEHASHEXISTS | 4x | 126 | 504 |
| EMITNOTEHASH | 15x | 1,285 | 19,275 |
| NULLIFIEREXISTS | 7x | 132 | 924 |
| EMITNULLIFIER | 20x | 1,540 | 30,800 |
| L1TOL2MSGEXISTS | 5x | 108 | 540 |
| SENDL2TOL1MSG | 2x | 209 | 418 |
| CALL | 3x | 3,312 | 9,936 |
| STATICCALL | 3x | 3,312 | 9,936 |
| GETCONTRACTINSTANCE | 4x | 1,527 | 6,108 |
| POSEIDON2 | 15x | 24 | 360 |
| ECADD | 10x | 27 | 270 |
| Opcode | Multiplier | Previous Cost | New Cost |
| ------------------- | ---------- | ------------- | -------- |
| FDIV | 25x | 9 | 225 |
| SLOAD | 10x | 129 | 1,290 |
| SSTORE | 20x | 1,657 | 33,140 |
| NOTEHASHEXISTS | 4x | 126 | 504 |
| EMITNOTEHASH | 15x | 1,285 | 19,275 |
| NULLIFIEREXISTS | 7x | 132 | 924 |
| EMITNULLIFIER | 20x | 1,540 | 30,800 |
| L1TOL2MSGEXISTS | 5x | 108 | 540 |
| SENDL2TOL1MSG | 2x | 209 | 418 |
| CALL | 3x | 3,312 | 9,936 |
| STATICCALL | 3x | 3,312 | 9,936 |
| GETCONTRACTINSTANCE | 4x | 1,527 | 6,108 |
| POSEIDON2 | 15x | 24 | 360 |
| ECADD | 10x | 27 | 270 |

**Impact**: Contracts with public bytecode performing any of these operations will see increased gas consumption.

Expand Down
40 changes: 10 additions & 30 deletions yarn-project/archiver/src/archiver/archiver_store_test_suite.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2276,17 +2276,13 @@ export function describeArchiverDataStore(
[
expect.objectContaining({
blockNumber: 2,
blockHash: L2BlockHash.fromField(await logsCheckpoints[2 - 1].checkpoint.blocks[0].header.hash()),
log: makePrivateLog(tags[0]),
isFromPublic: false,
logData: makePrivateLog(tags[0]).getEmittedFields(),
}),
],
[
expect.objectContaining({
blockNumber: 1,
blockHash: L2BlockHash.fromField(await logsCheckpoints[1 - 1].checkpoint.blocks[0].header.hash()),
log: makePrivateLog(tags[1]),
isFromPublic: false,
logData: makePrivateLog(tags[1]).getEmittedFields(),
}),
],
]);
Expand All @@ -2312,15 +2308,11 @@ export function describeArchiverDataStore(
[
expect.objectContaining({
blockNumber: 1,
blockHash: L2BlockHash.fromField(await logsCheckpoints[1 - 1].checkpoint.blocks[0].header.hash()),
log: makePrivateLog(tags[0]),
isFromPublic: false,
logData: makePrivateLog(tags[0]).getEmittedFields(),
}),
expect.objectContaining({
blockNumber: newBlockNumber,
blockHash: L2BlockHash.fromField(await newCheckpoint.checkpoint.blocks[0].header.hash()),
log: newLog,
isFromPublic: false,
logData: newLog.getEmittedFields(),
}),
],
]);
Expand All @@ -2338,9 +2330,7 @@ export function describeArchiverDataStore(
[
expect.objectContaining({
blockNumber: 1,
blockHash: L2BlockHash.fromField(await logsCheckpoints[1 - 1].checkpoint.blocks[0].header.hash()),
log: makePrivateLog(tags[1]),
isFromPublic: false,
logData: makePrivateLog(tags[1]).getEmittedFields(),
}),
],
]);
Expand Down Expand Up @@ -2425,17 +2415,13 @@ export function describeArchiverDataStore(
[
expect.objectContaining({
blockNumber: 2,
blockHash: L2BlockHash.fromField(await logsCheckpoints[2 - 1].checkpoint.blocks[0].header.hash()),
log: makePublicLog(tags[0]),
isFromPublic: true,
logData: makePublicLog(tags[0]).getEmittedFields(),
}),
],
[
expect.objectContaining({
blockNumber: 1,
blockHash: L2BlockHash.fromField(await logsCheckpoints[1 - 1].checkpoint.blocks[0].header.hash()),
log: makePublicLog(tags[1]),
isFromPublic: true,
logData: makePublicLog(tags[1]).getEmittedFields(),
}),
],
]);
Expand All @@ -2461,15 +2447,11 @@ export function describeArchiverDataStore(
[
expect.objectContaining({
blockNumber: 1,
blockHash: L2BlockHash.fromField(await logsCheckpoints[1 - 1].checkpoint.blocks[0].header.hash()),
log: makePublicLog(tags[0]),
isFromPublic: true,
logData: makePublicLog(tags[0]).getEmittedFields(),
}),
expect.objectContaining({
blockNumber: newBlockNumber,
blockHash: L2BlockHash.fromField(await newCheckpoint.checkpoint.blocks[0].header.hash()),
log: newLog,
isFromPublic: true,
logData: newLog.getEmittedFields(),
}),
],
]);
Expand All @@ -2487,9 +2469,7 @@ export function describeArchiverDataStore(
[
expect.objectContaining({
blockNumber: 1,
blockHash: L2BlockHash.fromField(await logsCheckpoints[1 - 1].checkpoint.blocks[0].header.hash()),
log: makePublicLog(tags[1]),
isFromPublic: true,
logData: makePublicLog(tags[1]).getEmittedFields(),
}),
],
]);
Expand Down
42 changes: 18 additions & 24 deletions yarn-project/archiver/src/archiver/kv_archiver_store/log_store.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
import { INITIAL_L2_BLOCK_NUM, MAX_NOTE_HASHES_PER_TX } from '@aztec/constants';
import { INITIAL_L2_BLOCK_NUM } from '@aztec/constants';
import { BlockNumber } from '@aztec/foundation/branded-types';
import { Fr } from '@aztec/foundation/curves/bn254';
import { createLogger } from '@aztec/foundation/log';
Expand Down Expand Up @@ -57,21 +57,16 @@ export class LogStore {
* @param block - The L2 block to extract logs from.
* @returns An object containing the private and public tagged logs for the block.
*/
async #extractTaggedLogsFromBlock(block: L2BlockNew) {
Copy link
Contributor Author

@benesjan benesjan Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

blockHash was dropped from TxScopedL2Log which results in this no longer needing to be async.

const blockHash = L2BlockHash.fromField(await block.hash());
#extractTaggedLogsFromBlock(block: L2BlockNew) {
// SiloedTag (as string) -> array of log buffers.
const privateTaggedLogs = new Map<string, Buffer[]>();
// "{contractAddress}_{tag}" (as string) -> array of log buffers.
const publicTaggedLogs = new Map<string, Buffer[]>();
const dataStartIndexForBlock =
block.header.state.partial.noteHashTree.nextAvailableLeafIndex -
block.body.txEffects.length * MAX_NOTE_HASHES_PER_TX;

block.body.txEffects.forEach((txEffect, txIndex) => {
block.body.txEffects.forEach(txEffect => {
const txHash = txEffect.txHash;
const dataStartIndexForTx = dataStartIndexForBlock + txIndex * MAX_NOTE_HASHES_PER_TX;

txEffect.privateLogs.forEach((log, logIndex) => {
txEffect.privateLogs.forEach(log => {
// Private logs use SiloedTag (already siloed by kernel)
const tag = log.fields[0];
this.#log.debug(`Found private log with tag ${tag.toString()} in block ${block.number}`);
Expand All @@ -80,18 +75,17 @@ export class LogStore {
currentLogs.push(
new TxScopedL2Log(
txHash,
dataStartIndexForTx,
logIndex,
block.number,
blockHash,
block.timestamp,
log,
log.getEmittedFields(),
txEffect.noteHashes,
txEffect.nullifiers[0],
).toBuffer(),
);
privateTaggedLogs.set(tag.toString(), currentLogs);
});

txEffect.publicLogs.forEach((log, logIndex) => {
txEffect.publicLogs.forEach(log => {
// Public logs use Tag directly (not siloed) and are stored with contract address
const tag = log.fields[0];
const contractAddress = log.contractAddress;
Expand All @@ -104,12 +98,11 @@ export class LogStore {
currentLogs.push(
new TxScopedL2Log(
txHash,
dataStartIndexForTx,
logIndex,
block.number,
blockHash,
block.timestamp,
log,
log.getEmittedFields(),
txEffect.noteHashes,
txEffect.nullifiers[0],
).toBuffer(),
);
publicTaggedLogs.set(key, currentLogs);
Expand All @@ -125,10 +118,11 @@ export class LogStore {
* @returns A map from tag (as string) to an array of serialized private logs belonging to that tag, and a map from
* "{contractAddress}_{tag}" (as string) to an array of serialized public logs belonging to that key.
*/
async #extractTaggedLogs(
blocks: L2BlockNew[],
): Promise<{ privateTaggedLogs: Map<string, Buffer[]>; publicTaggedLogs: Map<string, Buffer[]> }> {
const taggedLogsInBlocks = await Promise.all(blocks.map(block => this.#extractTaggedLogsFromBlock(block)));
#extractTaggedLogs(blocks: L2BlockNew[]): {
privateTaggedLogs: Map<string, Buffer[]>;
publicTaggedLogs: Map<string, Buffer[]>;
} {
const taggedLogsInBlocks = blocks.map(block => this.#extractTaggedLogsFromBlock(block));

// Now we merge the maps from each block into a single map.
const privateTaggedLogs = taggedLogsInBlocks.reduce((acc, { privateTaggedLogs }) => {
Expand All @@ -155,8 +149,8 @@ export class LogStore {
* @param blocks - The blocks for which to add the logs.
* @returns True if the operation is successful.
*/
async addLogs(blocks: L2BlockNew[]): Promise<boolean> {
const { privateTaggedLogs, publicTaggedLogs } = await this.#extractTaggedLogs(blocks);
addLogs(blocks: L2BlockNew[]): Promise<boolean> {
const { privateTaggedLogs, publicTaggedLogs } = this.#extractTaggedLogs(blocks);

const keysOfPrivateLogsToUpdate = Array.from(privateTaggedLogs.keys());
const keysOfPublicLogsToUpdate = Array.from(publicTaggedLogs.keys());
Expand Down
110 changes: 6 additions & 104 deletions yarn-project/pxe/src/logs/log_service.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,10 @@
import { BlockNumber } from '@aztec/foundation/branded-types';
import { randomInt } from '@aztec/foundation/crypto/random';
import { Fr } from '@aztec/foundation/curves/bn254';
import { KeyStore } from '@aztec/key-store';
import { openTmpStore } from '@aztec/kv-store/lmdb-v2';
import { AztecAddress } from '@aztec/stdlib/aztec-address';
import { L2BlockHash } from '@aztec/stdlib/block';
import type { AztecNode } from '@aztec/stdlib/interfaces/server';
import { PublicLog, Tag, TxScopedL2Log } from '@aztec/stdlib/logs';
import { TxHash, randomIndexedTxEffect } from '@aztec/stdlib/tx';
import { Tag } from '@aztec/stdlib/logs';
import { randomTxScopedPrivateL2Log } from '@aztec/stdlib/testing';

import { type MockProxy, mock } from 'jest-mock-extended';

Expand Down Expand Up @@ -70,18 +67,12 @@ describe('LogService', () => {
});

it('returns a public log if one is found', async () => {
const scopedLog = await TxScopedL2Log.random(true);
(scopedLog.log as PublicLog).contractAddress = contractAddress;
const scopedLog = randomTxScopedPrivateL2Log();

aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[scopedLog]]);
aztecNode.getPrivateLogsByTags.mockResolvedValue([[]]);
const indexedTxEffect = await randomIndexedTxEffect();

aztecNode.getTxEffect.mockImplementation((txHash: TxHash) =>
txHash.equals(scopedLog.txHash) ? Promise.resolve(indexedTxEffect) : Promise.resolve(undefined),
);

const request = new LogRetrievalRequest(contractAddress, new Tag(scopedLog.log.fields[0]));
const request = new LogRetrievalRequest(contractAddress, new Tag(scopedLog.logData[0]));

const responses = await logService.bulkRetrieveLogs([request]);

Expand All @@ -90,106 +81,17 @@ describe('LogService', () => {
});

it('returns a private log if one is found', async () => {
const scopedLog = await TxScopedL2Log.random(false);
const scopedLog = randomTxScopedPrivateL2Log();

aztecNode.getPrivateLogsByTags.mockResolvedValue([[scopedLog]]);
aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[]]);
const indexedTxEffect = await randomIndexedTxEffect();
aztecNode.getTxEffect.mockResolvedValue(indexedTxEffect);

aztecNode.getTxEffect.mockImplementation((txHash: TxHash) =>
txHash.equals(scopedLog.txHash) ? Promise.resolve(indexedTxEffect) : Promise.resolve(undefined),
);

const request = new LogRetrievalRequest(contractAddress, new Tag(scopedLog.log.fields[0]));
const request = new LogRetrievalRequest(contractAddress, new Tag(scopedLog.logData[0]));

const responses = await logService.bulkRetrieveLogs([request]);

expect(responses.length).toEqual(1);
expect(responses[0]).not.toBeNull();
});
});

describe('getPublicLogByTag', () => {
Copy link
Contributor Author

@benesjan benesjan Dec 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made the getPublicLogByTag function private in the log service and it got greatly simplified by this PR that I don't think we need these tests anymore.

const tag = new Tag(Fr.random());

beforeEach(() => {
aztecNode.getPublicLogsByTagsFromContract.mockReset();
aztecNode.getTxEffect.mockReset();
});

it('returns null if no logs found for tag', async () => {
aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[]]);

const result = await logService.getPublicLogByTag(tag, contractAddress);
expect(result).toBeNull();
});

it('returns log data when single log found', async () => {
const scopedLog = await TxScopedL2Log.random(true);
const logContractAddress = (scopedLog.log as PublicLog).contractAddress;

aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[scopedLog]]);
const indexedTxEffect = await randomIndexedTxEffect();
aztecNode.getTxEffect.mockImplementation((txHash: TxHash) =>
txHash.equals(scopedLog.txHash) ? Promise.resolve(indexedTxEffect) : Promise.resolve(undefined),
);

const result = (await logService.getPublicLogByTag(tag, logContractAddress))!;

expect(result.logPayload).toEqual(scopedLog.log.getEmittedFieldsWithoutTag());
expect(result.uniqueNoteHashesInTx).toEqual(indexedTxEffect.data.noteHashes);
expect(result.txHash).toEqual(scopedLog.txHash);
expect(result.firstNullifierInTx).toEqual(indexedTxEffect.data.nullifiers[0]);

expect(aztecNode.getPublicLogsByTagsFromContract).toHaveBeenCalledWith(logContractAddress, [tag]);
expect(aztecNode.getTxEffect).toHaveBeenCalledWith(scopedLog.txHash);
});

it('throws if multiple logs found for tag', async () => {
const scopedLog = await TxScopedL2Log.random(true);
aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[scopedLog, scopedLog]]);
const logContractAddress = (scopedLog.log as PublicLog).contractAddress;

await expect(logService.getPublicLogByTag(tag, logContractAddress)).rejects.toThrow(/Got 2 logs for tag/);
});

it('throws if tx effect not found', async () => {
const scopedLog = await TxScopedL2Log.random(true);
aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[scopedLog]]);
aztecNode.getTxEffect.mockResolvedValue(undefined);
const logContractAddress = (scopedLog.log as PublicLog).contractAddress;

await expect(logService.getPublicLogByTag(tag, logContractAddress)).rejects.toThrow(
/failed to retrieve tx effects/,
);
});

it('returns log fields that are actually emitted', async () => {
const logContractAddress = await AztecAddress.random();
const logPlaintext = [Fr.random()];
const logContent = [tag.value, ...logPlaintext];

const log = PublicLog.from({
contractAddress: logContractAddress,
fields: logContent,
});
const scopedLogWithPadding = new TxScopedL2Log(
TxHash.random(),
randomInt(100),
randomInt(100),
BlockNumber(randomInt(100)),
L2BlockHash.random(),
0n,
log,
);

aztecNode.getPublicLogsByTagsFromContract.mockResolvedValue([[scopedLogWithPadding]]);
aztecNode.getTxEffect.mockResolvedValue(await randomIndexedTxEffect());

const result = await logService.getPublicLogByTag(tag, logContractAddress);

expect(result?.logPayload).toEqual(logPlaintext);
});
});
});
Loading
Loading