@mcroomp, unfortunately this is not working, not enough precision for all u16 range https://github.com/microsoft/lepton_jpeg_rust/blob/2f8c32420e3c67462afa459ece898b001794ce2e/src/structs/quantization_tables.rs#L88 Even 2^32 fails - first at 17215. A classical scheme from Warren should work, but it has variable shifts.
@mcroomp, unfortunately this is not working, not enough precision for all u16 range
lepton_jpeg_rust/src/structs/quantization_tables.rs
Line 88 in 2f8c324
Even 2^32 fails - first at 17215. A classical scheme from Warren should work, but it has variable shifts.