Skip to content

how to convert yoloNAS model to int8? #2598

Closed Answered by alexsu52
MinGiSa asked this question in Q&A
Discussion options

You must be logged in to vote

Hi @MinGiSa,

NNCF does not support quantization of custom PyTorch modules with weights. For example, "yolo_nas_l" has NDFLHeads module which calls conv2d function with self.proj_conv in forward which can not be quantized automaticly: https://github.com/Deci-AI/super-gradients/blob/7067736cb9062245aa4f118d91b03bf8de898ef7/src/super_gradients/training/models/detection_models/yolo_nas/dfl_heads.py#L210.

Taking into account that this module is the head of the model I would recommend to ignore it to preserve accuracy of the quantized model:

    quantized_model = nncf.quantize(
        torchModel,
        calibrationDataset,
        preset=nncf.QuantizationPreset.MIXED,
        ignored_scope=nn…

Replies: 1 comment 4 replies

Comment options

You must be logged in to vote
4 replies
@MinGiSa
Comment options

@alexsu52
Comment options

@jzdcf
Comment options

@alexsu52
Comment options

Answer selected by MinGiSa
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Category
Q&A
Labels
None yet
3 participants