[lib] Move type cast in Fletcher checksum

The int type cast should be on the whole expression
passed to the mod operator. Otherwise it won't work
when/if c0/c1 is unsigned.
Making c0/c1 unsigned makes it possible to use 5802
as MODX value.

diff --git a/lib/checksum.c b/lib/checksum.c
index f6d74d31201bd2a7bd0484094910ac6093e430fa..3ddde815234a554109cdbf9be8b5e93abf46a2ed 100644 (file)
--- a/lib/checksum.c
+++ b/lib/checksum.c
@@ -91,7 +91,7 @@ fletcher_checksum(u_char * buffer, const size_t len, const uint16_t offset)
     }
   
   /* The cast is important, to ensure the mod is taken as a signed value. */
-  x = ((int)(len - offset - 1) * c0 - c1) % 255;
+  x = (int)((len - offset - 1) * c0 - c1) % 255;
 
   if (x <= 0)
     x += 255;