Aggregate State And Merge

Generic Description

Use aggregate state and merge forms when you need to store partial aggregate state now and combine it later without keeping the full raw input stream around.

Simple example:

MATCH (u:User)-[:HAS_APP_SESSION]->(s:AppSession)
WITH u.profile.location.country AS country,
     avg_state(s.duration_seconds) AS duration_state
RETURN country, avg_merge(duration_state) AS avg_duration

Consumer-Level Explanation

State and merge forms matter when you want aggregation to be composable.

Instead of only asking for the final answer, you ask for a mergeable partial answer. That is useful for:

• partial aggregation
• grouped spill-backed execution
• staging analytics across pipeline boundaries
• materialized summaries that do not want to keep every raw row forever

The current implementation now supports broad state and merge coverage across:

• basic scalar aggregates
• distinct and approximate distinct aggregates
• collection and heavy-hitter aggregates
• percentile, statistical, covariance, correlation, and regression families

More Detailed Explanation

This surface exists because “aggregate now, combine later” is a core optimization primitive.

Without explicit state and merge forms, systems usually end up doing one of two bad things:

• recomputing from raw data too often
• inventing undocumented internal summary formats that users cannot reason about

Nexyron exposes the idea directly in Cypher through forms such as:

• count_state / count_merge
• sum_state / sum_merge
• avg_state / avg_merge
• top_k_state / top_k_merge
• percentile, statistical, covariance, correlation, and regression state/merge pairs

That design also lines up with the current spill-backed executor, which can keep grouped workloads exact on the supported subset by carrying mathematically sufficient partial state rather than inventing approximate shortcuts.

Advanced Example

MATCH (u:User)-[e:VIEWED]->(d:Document)
TIME e.ts BETWEEN datetime('2025-01-01T00:00:00Z') AND datetime('2025-02-01T00:00:00Z')
WITH date_trunc('day', e.ts) AS day_bucket,
     avgstate(properties(d).prize) AS partial_state
RETURN day_bucket,
       avgmerge(partial_state) AS merged_metric
ORDER BY day_bucket

Real Use Cases

• partial grouped analytics where intermediate states move between stages
• spill-backed execution for large grouped workloads
• rollup pipelines that want mergeable summaries instead of full raw row retention
• operational and social analytics where top-k, percentiles, and regression metrics need to be recombined exactly

Real Limitations And Tradeoffs

• mergeable state is powerful, but it is only useful when the state format remains mathematically faithful to the target aggregate
• state values are not meant to be human-friendly business reports; they are execution-oriented summaries
• order-sensitive aggregates remain more subtle than pure scalar summaries, especially when users expect globally meaningful ordering semantics