Skip to content

Conversation

QYuFong
Copy link
Contributor

@QYuFong QYuFong commented Jul 17, 2025

What does this PR do?

Fixes # (issue)

Adds # (feature)

Before submitting

  • This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
  • Did you read the contributor guideline?
  • Did you make sure to update the documentation with your changes? E.g. record bug fixes or new features in What's New. Here are the
    documentation guidelines
  • Did you build and run the code without any errors?
  • Did you report the running environment (NPU type/MS version) and performance in the doc? (better record it for data loading, model inference, or training tasks)
  • Did you write any new necessary tests?

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@xxx

@hadipash
Copy link
Collaborator

Update here too?

jit_level: Optional[Literal["O0", "O1", "O2"]] = None,

@SamitHuang
Copy link
Collaborator

/gemini review

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adapts the code for MindSpore versions 2.6 and 2.7. The changes mainly involve updating the JIT optimization level and replacing deprecated functions like chunk_ext with mint.chunk.

I've found a critical issue in examples/opensora_hpcai/scripts/inference.py where the performance metric for sampling speed is calculated incorrectly. This leads to misleading logs. I've provided a suggestion to fix the formula and use the correct variable for batch size. The other changes look good and are consistent with the goal of adapting to a new MindSpore version.

batch_time = time.time() - start_time
logger.info(
f"Batch time cost: {batch_time:.3f}s, sampling speed: {args.sampling_steps * ns / batch_time:.4f} step/s"
f"Batch time cost: {batch_time:.3f}s, sampling speed: {batch_time / args.sampling_steps * ns:.4f} s/step"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The formula for calculating sampling speed in s/step is incorrect. This will result in misleading performance metrics being logged.

There are two issues with the current calculation:

  1. Incorrect formula: To calculate seconds per step (s/step), you should divide the total time by the total number of steps. The current formula batch_time / args.sampling_steps * ns is evaluated as (batch_time / args.sampling_steps) * ns due to operator precedence, which is not mathematically correct for s/step. It should be batch_time / (args.sampling_steps * num_samples).

  2. Incorrect number of samples: The calculation uses ns, which is the total number of prompts for the entire run. For a per-batch performance metric, it should use the number of samples in the current batch. You can get this from samples.shape[0], as samples is available in this scope.

I've suggested a fix below that addresses both points.

Suggested change
f"Batch time cost: {batch_time:.3f}s, sampling speed: {batch_time / args.sampling_steps * ns:.4f} s/step"
f"Batch time cost: {batch_time:.3f}s, sampling speed: {batch_time / (args.sampling_steps * samples.shape[0]):.4f} s/step"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@QYuFong Pls fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@QYuFong
Copy link
Contributor Author

QYuFong commented Jul 21, 2025

Update here too?

jit_level: Optional[Literal["O0", "O1", "O2"]] = None,

Just sd3 need to change O2 -> O1, because SD3 has discontinuous input, the GE backend no longer supports it. Other models are not affected for the time being.

@QYuFong QYuFong force-pushed the master_adapted_new_api branch from d839b94 to 2142611 Compare July 21, 2025 11:51
@hadipash
Copy link
Collaborator

Just sd3 need to change O2 -> O1, because SD3 has discontinuous input, the GE backend no longer supports it. Other models are not affected for the time being.

I mean update typing and docs: jit_level: Optional[Literal["O0", "O1", "O2"]] = None -> jit_level: Optional[Literal["O0", "O1"]] = None

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants