Skip to content

Optimized ONNX Transform via Class Merging and Thread Pooling #546

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
wants to merge 6 commits into from
Closed

Optimized ONNX Transform via Class Merging and Thread Pooling #546

wants to merge 6 commits into from

Conversation

abhishek-singh591
Copy link
Contributor

@abhishek-singh591 abhishek-singh591 commented Aug 23, 2025 •
edited
Loading

Optimized ONNX Transform via Class Merging and Thread Pooling

This PR follows up on #539 – Optimized ONNX transform class via multithreading.

It merges the FP16 and Split ONNX transform classes into a single implementation to eliminate redundant tensor loading and iteration. Additionally, the transform logic has been refactored to use a thread pool, replacing the previous sequential loop to parallelize tensor operations.

Performance Benchmarks:-

Model Original Duration (s) Optimized Duration (s)
LLaMA 3.1 8B 88.35 58.55
LLaMA 3.1 70B 1029.82 727.37

Note: Thread count is set to os.cpu_count() * 4 to better handle I/O-bound workloads. Performance may vary depending on system hardware and threading capabilities.

ochougul
ochougul requested changes Sep 1, 2025
View reviewed changes
ochougul
ochougul requested changes Sep 4, 2025
View reviewed changes
Copy link
Contributor

@ochougul ochougul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. can merge if CI is passing

@abhishek-singh591 abhishek-singh591 force-pushed the optimized_onnx_tranform branch 2 times, most recently from 00fcaf2 to 9b9c41d Compare September 5, 2025 11:12
@abhishek-singh591
merged fp16 and split in onnx transform
6361518 
Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
@abhishek-singh591
Add warning when both flags apply_split and apply_clip are false and …
e957632 
…added removed comments

Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
@abhishek-singh591
Add warning when both flags apply_split and apply_clip are false and …
a9b01c3 
…added removed comments

Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
@abhishek-singh591
Fixed transform flag and other importing related issue
cfc4809 
Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
@abhishek-singh591
Deleted run.py file
6910ece 
Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
@abhishek-singh591
Modified the default model_name argument
f64b429 
Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>
quic-rishinr
quic-rishinr requested changes Sep 10, 2025
View reviewed changes
@abhishek-singh591 abhishek-singh591 mentioned this pull request Oct 9, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@quic-rishinr quic-rishinr quic-rishinr requested changes

@quic-hemagnih quic-hemagnih Awaiting requested review from quic-hemagnih quic-hemagnih is a code owner

@quic-amitraj quic-amitraj Awaiting requested review from quic-amitraj quic-amitraj is a code owner

@ochougul ochougul Awaiting requested review from ochougul ochougul is a code owner

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@abhishek-singh591 @ochougul @quic-rishinr