Skip to content

feat(shredding): support selected-key pushdown by MapSharedShreddingAccessBuilder - #195

Merged
SteNicholas merged 5 commits into
apache:mainfrom
lszskye:sync_shared_shredding
Aug 13, 2026
Merged

feat(shredding): support selected-key pushdown by MapSharedShreddingAccessBuilder#195
SteNicholas merged 5 commits into
apache:mainfrom
lszskye:sync_shared_shredding

Conversation

@lszskye

@lszskye lszskye commented Aug 11, 2026

Copy link
Copy Markdown
Collaborator

Purpose

Support projecting selected keys from a top-level shared-shredding MAP column as a STRUCT.

This change:

  • Adds MapSharedShreddingAccessBuilder for constructing selected-key STRUCT read fields;
  • introduces MapFieldReadPlan to describe the logical output field, physical read field, and materialization behavior;

Tests

src/paimon/core/utils/nested_projection_utils_test.cpp
src/paimon/common/data/shredding/map_shared_shredding_schema_utils_test.cpp
src/paimon/common/data/shredding/map_shared_shredding_file_reader_test.cpp
test/inte/write_and_read_inte_test.cpp

API and Format

This change adds the public MapSharedShreddingAccessBuilder API under include/paimon.

@lszskye
lszskye force-pushed the sync_shared_shredding branch from 6ac023a to bba842b Compare August 11, 2026 08:58
@lszskye
lszskye requested a review from lxy-9602 August 11, 2026 09:41
Comment thread src/paimon/common/data/shredding/map_shared_shredding_file_reader.cpp Outdated
Comment thread src/paimon/common/data/shredding/map_shared_shredding_file_reader.cpp Outdated
Comment thread src/paimon/core/io/field_mapping_reader.cpp
Comment thread src/paimon/core/operation/internal_read_context.cpp
Comment thread src/paimon/core/operation/internal_read_context.cpp
Comment thread src/paimon/core/utils/nested_projection_utils.h
Comment thread test/inte/write_and_read_inte_test.cpp

@lxy-9602 lxy-9602 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@SteNicholas SteNicholas left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@SteNicholas
SteNicholas merged commit 67e6aca into apache:main Aug 13, 2026
28 of 29 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants