[Pipe] Optimize OPC UA TsFile last-value transfer - #18433
Open
Caideyipi wants to merge 3 commits into
Open
Conversation
jt2594838
reviewed
Aug 10, 2026
Comment on lines
+608
to
+610
| if (Objects.isNull(timeseriesDataTypes)) { | ||
| throw new IOException(); | ||
| } |
Contributor
There was a problem hiding this comment.
Add clearer message. Or return null instead.
Collaborator
Author
There was a problem hiding this comment.
Applied in 7eca368 using the suggested null option: device/type/count metadata inconsistencies now return null and immediately trigger the existing Tablet fallback instead of throwing a message-less IOException.
jt2594838
approved these changes
Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Optimize the OPC UA sink's client-server TsFile path by reading each timeseries' last point directly from TsFile metadata instead of materializing every Tablet.
Metadata fast path
TsFileLastReader; only BLOB/OBJECT-style types without values in statistics need their last chunk read.TimeValuePair, so logical types such as TIMESTAMP, STRING, and BLOB are not degraded byTsPrimitiveType.TsFileLastReader.hasNext()logs and suppresses IOExceptions.Compatibility and event semantics
Tests
Manual performance result
Measured locally on 2026-08-11 with Windows 11 64-bit, an ASUS Zenbook UX3404VC (Intel Core i9-13900H, 14 cores/20 threads), JDK 17.0.15, and Maven 3.8.8.
Command:
Workload: 2 devices, 32 measurements/device (1 BLOB measurement), 50,000 rows/device, 3,200,000 points, and a 12.32 MiB TsFile; 1 warmup iteration and 50 measured iterations per round across 5 alternating rounds. Three independent JVM runs were completed; the table reports the median across those runs (each run reports its own 5-round median).
Observed CPU speedup across the three runs was 166.67x-254.50x; allocation reduction stayed at 99.8%, and peak-heap reduction was 94.0%-94.6%. The benchmark's final-value maps were equal in every run, and each run passed the JUnit correctness check (
Tests run: 1, Failures: 0, Errors: 0).This PR has:
Key changed/added classes
OpcUaSinkIoTDBOpcUaClientOpcUaNameSpaceOpcUaSinkTsFileMetadataTestOpcUaNameSpaceMetadataTest