librsync  2.3.1
NEWS.md
1 # librsync NEWS
2 
3 ## librsync 2.3.1
4 
5 Released 2020-05-19
6 
7  * Fix #198 cmake popt detection using pkg-config and #199 test scripts on
8  FreeBSD. Fixes and tidies FindPOPT.cmake and Findlibb2.cmake to use
9  pkg-config correctly and behave more like official FindPackage() cmake
10  modules. Makes all test scripts use /bin/sh instead of /bin/bash. (dbaarda,
11  mandree https://github.com/librsync/librsync/pull/200)
12 
13  * Change default block_len to always be a multiple of the blake2b 128 byte
14  blocksize for efficiency. Tidy and update docs to explain using
15  rs_sig_args() and rs_build_hash_table(), add rs_file_*() utils, and
16  document new magic types. Remove really obsolete entries in TODO.md. Update
17  to Doxygen 1.8.16. (dbaarda, https://github.com/librsync/librsync/pull/195)
18 
19  * Improve hashtable performance by adding a small optional bloom filter,
20  reducing max loadfactor from 80% to 70%, Fix hashcmp_count stats to include
21  comparing against empty buckets. This speeds up deltas by 20%~50%.
22  (dbaarda, https://github.com/librsync/librsync/pull/192,
23  https://github.com/librsync/librsync/pull/193,
24  https://github.com/librsync/librsync/pull/196)
25 
26  * Optimize rabinkarp_update() by correctly using unsigned constants and
27  manually unrolling the loop for best performance. (dbaarda,
28  https://github.com/librsync/librsync/pull/191)
29 
30 ## librsync 2.3.0
31 
32 Released 2020-04-07
33 
34  * Bump minor version from 2.2.1 to 2.3.0 to reflect additional rs_sig_args()
35  and strong_len=-1 support.
36 
37  * Add public rs_sig_args() function for getting the recommend signature args
38  from the file size. Added support to rdiff for `--sum-size=-1` to indicate
39  "use minimum size safe against random block collisions". Added warning
40  output for sum-sizes that are too small to be safe. Fixed possible rdiff
41  bug affecting popt parsing on non-little-endian platforms. (dbaarda,
42  https://github.com/librsync/librsync/pull/109)
43 
44  * Fixed yet more compiler warnings for various platforms/compilers.
45  (Adsun701, texierp, https://github.com/librsync/librsync/pull/187,
46  https://github.com/librsync/librsync/pull/188)
47 
48  * Improved cmake popt handling to find popt dependencies using PkgConfig.
49  (ffontaine, https://github.com/librsync/librsync/pull/186)
50 
51  * Tidied internal code and improved tests for netint.[ch], tube.c, and
52  hashtable.h. (dbaarda, https://github.com/librsync/librsync/pull/183
53  https://github.com/librsync/librsync/pull/185).
54 
55  * Improved C99 compatibility. Add `-std=c99 -pedantic` to `CMAKE_C_FLAGS` for
56  gcc and clang. Fix all C99 warnings by making all code C99 compliant. Tidy
57  all CMake checks, `#cmakedefines`, and `#includes`. Fix 64bit support for
58  mdfour checksums (texierp, dbaarda,
59  https://github.com/librsync/librsync/pull/181,
60  https://github.com/librsync/librsync/pull/182)
61 
62  * Usage clarified in rdiff (1) man page. (AaronM04,
63  https://github.com/librsync/librsync/pull/180)
64 
65 ## librsync 2.2.1
66 
67 Released 2019-10-16
68 
69  * Fix #176 hangs calculating deltas for files larger than 4GB. (dbaarda,
70  https://github.com/librsync/librsync/pull/177)
71 
72 ## librsync 2.2.0
73 
74 Released 2019-10-12
75 
76  * Bump minor version from 2.1.0 to 2.2.0 to reflect additional RabinKarp
77  rollsum support.
78 
79  * Fix MSVC builds by adding missing LIBRSYNC_EXPORT to variables in
80  librsync.h, add -DLIBRSYNC_STATIC_DEFINE to the sumset_test target,
81  and correctly install .dll files in the bin directory.
82  (adsun701, https://github.com/librsync/librsync/pull/161)
83 
84  * Add RabinKarp rollsum support and make it the default. RabinKarp is a much
85  better rolling hash, which reduces the risk of hash collision corruption
86  and speeds up delta calculations. The rdiff cmd gets a new `-R
87  (rollsum|rabinkarp)` argument with the default being `rabinkarp`, Use `-R
88  rollsum` to generate backwards-compatible signatures. (dbaarda,
89  https://github.com/librsync/librsync/issues/3)
90 
91  * Use single-byte literal commands for small inserts in deltas. This makes
92  each small insert use 1 less byte in deltas. (dbaarda,
93  https://github.com/librsync/librsync/issues/120)
94 
95  * Fix multiple warnings (cross-)compiling for windows. (Adsun701,
96  https://github.com/librsync/librsync/pull/165,
97  https://github.com/librsync/librsync/pull/166)
98 
99  * Change rs_file_size() to report -1 instead of 0 for unknown file sizes (not
100  a regular file). (dbaarda https://github.com/librsync/librsync/pull/168)
101 
102  * Add cmake BUILD_SHARED_LIBS option for static library support.
103  BUILD_SHARED_LIBS defaults to ON, and can be set to OFF using `ccmake .` to
104  build librsync as a static library. (dbaarda
105  https://github.com/librsync/librsync/pull/169)
106 
107  * Fix compile errors and add .gitignore entries for MSVS 2019. Fixes
108  hashtable.h to be C99 compliant. (ardovm
109  https://github.com/librsync/librsync/pull/170)
110 
111 ## librsync 2.1.0
112 
113 Released 2019-08-19
114 
115  * Bump minor version from 2.0.3 to 2.1.0 to reflect additions to librsync.h.
116 
117  * Fix exporting of private symbols from librsync library. Add export of
118  useful large file functions `rs_file_open()`, `rs_file_close()`, and
119  `rs_file_size()` to librsync.h. Add export of `rs_signature_log_stats()` to
120  log signature hashtable hit/miss stats. Improve rdiff error output.
121  (dbaarda, https://github.com/librsync/librsync/issues/130)
122 
123  * Updated release process to include stable tarballs. (dbaarda,
124  https://github.com/librsync/librsync/issues/146)
125 
126  * Remove redundant and broken `--paranoia` argument from rdiff. (dbaarda,
127  https://github.com/librsync/librsync/issues/155)
128 
129  * Fix memory leak of `rs_signature_t->block_sigs` when freeing signatures.
130  (telles-simbiose, https://github.com/librsync/librsync/pull/147)
131 
132  * Document delta file format. (zmj,
133  https://github.com/librsync/librsync/issues/46)
134 
135  * Fix up doxygen comments. (dbaarda,
136  https://github.com/librsync/librsync/pull/151)
137 
138 ## librsync 2.0.2
139 
140 Released 2018-02-27
141 
142  * Improve CMake install paths configuration (wRAR,
143  https://github.com/librsync/librsync/pull/133) and platform support
144  checking when cross-compiling (fornwall,
145  https://github.com/librsync/librsync/pull/136).
146 
147  * Fix Unaligned memory access for rs_block_sig_init() (dbaarda,
148  https://github.com/librsync/librsync/issues/135).
149 
150  * Fix hashtable_test.c name collision for key_t in sys/types.h on some
151  platforms (dbaarda, https://github.com/librsync/librsync/issues/134)
152 
153  * Format code with consistent style, adding `make tidy` and `make
154  tidyc` targets for reformating code and comments. (dbaarda,
155  https://github.com/librsync/librsync/issues/125)
156 
157  * Removed perl as a build dependency. Note it is still required for some
158  tests. (dbaarda, https://github.com/librsync/librsync/issues/75)
159 
160  * Update RPM spec file for v2.0.2 and fix cmake man page install. (deajan,
161  https://github.com/librsync/librsync/issues/47)
162 
163 ## librsync 2.0.1
164 
165 Released 2017-10-17
166 
167  * Extensively reworked Doxygen documentation, now available at
168  http://librsync.sourcefrog.net/ (Martin Pool)
169 
170  * Removed some declarations from librsync.h that were unimplemented or no
171  longer ever useful: `rs_work_options`, `rs_accum_value`. Remove
172  declaration of unimplemented `rs_mdfour_file()`. (Martin Pool)
173 
174  * Remove shipped `snprintf` code: no longer acutally linked after changing to
175  CMake, and since it's part of C99 it should be widely available.
176  (Martin Pool)
177 
178  * Document that Ninja (http://ninja-build.org/) is supported under CMake.
179  It's a bit faster and nicer than Make. (Martin Pool)
180 
181  * `make check` (or `ninja check` etc) will now build and run the tests.
182  Previously due to a CMake limitation, `make test` would only run existing
183  tests and could fail if they weren't built.
184  (Martin Pool, https://github.com/librsync/librsync/issues/49)
185 
186  * Added cmake options to exclude rdiff target and compression from build.
187  See install documentation for details. Thanks to Michele Bertasi.
188 
189  * `popt` is only needed when `rdiff` is being built. (gulikoza)
190 
191  * Improved large file support for platforms using different variants
192  of `fseek` (`fseeko`, `fseeko64`, `_fseeki64`), `fstat` (`fstat64`,
193  `_fstati64`), and `fileno` (`_fileno`). (dbaarda, charlievieth,
194  gulikoza, marius-nicolae)
195 
196  * `rdiff -s` option now shows bytes read/written and speed. (gulikoza).
197  For delta operations it also shows hashtable match statistics. (dbaarda)
198 
199  * Running rdiff should not overwrite existing files (signatures, deltas and
200  new patched files) by default. If the destination file exists, rdiff will
201  now exit with an error. Add new option -f (--force) to overwrite existing
202  files. (gulikoza)
203 
204  * Improve signature memory allocation (doubling size instead of calling
205  realloc for every sig block) and added support for preallocation. See
206  streaming.md job->estimated_signature_count for usage when using the
207  library. `rdiff` uses this by default if possible. (gulikoza, dbaarda)
208 
209  * Significantly tidied signature handling code and testing, resulting in more
210  consistent error handling behaviour, and making it easier to plug in
211  alternative weak and strong sum implementations. Also fixed "slack delta"
212  support for delta calculation with no signature. (dbaarda)
213 
214  * `stdint.h` and `inttypes.h` from C99 is now required. Removed redundant
215  librsync-config.h header file. (dbaarda)
216 
217  * Lots of small fixes for windows platforms and building with MSVC.
218  (lasalvavida, mbrt, dbaarda)
219 
220  * New open addressing hashtable implementation that significantly speeds up
221  delta operations, particularly for large files. Also fixed degenerate
222  behaviour with large number of duplicate blocks like runs of zeros
223  in sparse files. (dbaarda)
224 
225  * Optional support with cmake option for using libb2 blake2 implementation.
226  Also updated included reference blake2 implementation with bug fixes
227  (dbaarda).
228 
229  * Improved default values for input and output buffer sizes. The defaults are
230  now --input-size=0 and --output-size=0, which will choose recommended
231  default sizes based on the --block-size and the operation being performed.
232  (dbaarda)
233 
234  * Fixed hanging for truncated input files. It will now correctly report an
235  error indicating an unexpected EOF was encountered. (dbaarda,
236  https://github.com/librsync/librsync/issues/32)
237 
238  * Fixed #13 so that faster slack delta's are used for signatures of
239  empty files. (dbaarda,
240  https://github.com/librsync/librsync/issues/13)
241 
242  * Fixed #33 so rs_job_iter() doesn't need calling twice with eof=1.
243  Also tidied and optimized it a bit. (dbaarda,
244  https://github.com/librsync/librsync/issues/33)
245 
246  * Fixed #55 remove excessive rs_fatal() calls, replacing checks for
247  programming errors with assert statements. Now rs_fatal() will only
248  be called for rare unrecoverable fatal errors like malloc failures or
249  impossibly large inputs. (dbaarda,
250  https://github.com/librsync/librsync/issues/55)
251 
252 ## librsync 2.0.0
253 
254 Released 2015-11-29
255 
256 Note: despite the major version bump, this release has few changes and should
257 be binary and API compatible with the previous version.
258 
259  * Bump librsync version number to 2.0, to match the library
260  soname/dylib version.
261  (Martin Pool, https://github.com/librsync/librsync/issues/48)
262 
263 ## librsync 1.0.1 (2015-11-21)
264 
265  * Better performance on large files. (VictorDenisov)
266 
267  * Add comment on usage of rs_build_hash_table(), and assert correct use.
268  Callers must call rs_build_hash_table() after loading the signature,
269  and before calling rs_delta_begin().
270  Thanks to Paul Harris <paulharris@computer.org>
271 
272  * Switch from autoconf to CMake.
273 
274  Thanks to Adam Schubert.
275 
276 ## librsync 1.0.0 (2015-01-23)
277 
278  * SECURITY: CVE-2014-8242: librsync previously used a truncated MD4
279  "strong" check sum to match blocks. However, MD4 is not cryptographically
280  strong. It's possible that an attacker who can control the contents of one
281  part of a file could use it to control other regions of the file, if it's
282  transferred using librsync/rdiff. For example this might occur in a
283  database, mailbox, or VM image containing some attacker-controlled data.
284 
285  To mitigate this issue, signatures will by default be computed with a
286  256-bit BLAKE2 hash. Old versions of librsync will complain about a
287  bad magic number when given these signature files.
288 
289  Backward compatibility can be obtained using the new
290  `rdiff sig --hash=md4`
291  option or through specifying the "signature magic" in the API, but
292  this should not be used when either the old or new file contain
293  untrusted data.
294 
295  Deltas generated from those signatures will also use BLAKE2 during
296  generation, but produce output that can be read by old versions.
297 
298  See https://github.com/librsync/librsync/issues/5
299 
300  Thanks to Michael Samuel <miknet.net> for reporting this and offering an
301  initial patch.
302 
303  * Various build fixes, thanks Timothy Gu.
304 
305  * Improved rdiff man page from Debian.
306 
307  * Improved librsync.spec file for building RPMs.
308 
309  * Fixed bug #1110812 'internal error: job made no progress'; on large
310  files.
311 
312  * Moved hosting to https://github.com/librsync/librsync/
313 
314  * Travis-CI.org integration test at https://travis-ci.org/librsync/librsync/
315 
316  * You can set `$LIBTOOLIZE` before running `autogen.sh`, for example on
317  OS X Homebrew where it is called `glibtoolize`.
318 
319 ## 0.9.7 (released 2004-10-10)
320 
321  * Yet more large file support fixes.
322 
323  * `extern "C"` guards in librsync.h to let it be used from C++.
324 
325  * Removed Debian files from dist tarball.
326 
327  * Changed rdiff to an installed program on "make install".
328 
329  * Refactored delta calculation code to be cleaner and faster.
330 
331  * \#879763: Fixed mdfour to work on little-endian machines which don't
332  like unaligned word access. This should make librsync work on
333  pa-risc, and it makes it slightly faster on ia64.
334 
335  * \#1022764: Fix corrupted encoding of some COPY commands in large
336  files.
337 
338  * \#1024881: Print long integers directly, rather than via casts to
339  double.
340 
341  * Fix printf formats for size_t: both the format and the argument
342  should be cast to long.
343 
344 ## 0.9.6
345 
346  * Large file support fixes.
347 
348  * [v]snprintf or _[v]snprintf autoconf replacement function fix.
349 
350  * Changed installed include file from rsync.h to librsync.h.
351 
352  * Migration to sourceforge for hosting.
353 
354  * Rollsum bugfix that produces much smaller deltas.
355 
356  * Memory leaks bugfix patches.
357 
358  * mdfour bigendian and >512M bugfix, plus optimisations patch.
359 
360  * autoconf/automake updates and cleanups for autoconf 2.53.
361 
362  * Windows compilation patch, heavily modified.
363 
364  * MacOSX compilation patch, modified to autoconf vararg macro fix.
365 
366  * Debian package build scripts patch.
367 
368 ## 0.9.5
369 
370  * Bugfix patch from Shirish Hemant Phatak
371 
372 ## 0.9.4: (library 1.1.0)
373 
374  * Fixes for rsync.h from Thorsten Schuett <thorsten.schuett@zib.de>
375 
376  * RLL encoding fix from Shirish Hemant Phatak <shirish@nustorage.com>
377 
378  * RPM spec file by Peter J. Braam <braam@clusterfs.com>
379 
380  * No (intentional) changes to binary API.
381 
382 ## 0.9.3
383 
384  * Big speed improvements in MD4 routines and generation of weak
385  checksums.
386 
387  * Patch to build on FreeBSD by Jos Backus <josb@cncdsl.com>
388 
389  * Suggestions to build on Solaris 2.6 from Alberto Accomazzi
390  <aaccomazzi@cfa.harvard.edu>
391 
392  * Add rs_job_drive, a generic mechanism for turning the library into
393  blocking mode. rs_whole_run now builds on top of this. The
394  filebuf interface has changed a little to accomodate it.
395 
396  * Generating and loading signatures now generates statistics.
397 
398  * More test cases.
399 
400  * I suspect there may be a bug in rolling checksums, but it probably
401  only causes inefficiency and not corruption.
402 
403  * Portability fixes for alphaev67-dec-osf5.1; at the moment builds
404  but does not work because librsync tries to do unaligned accesses.
405 
406  * Works on sparc64-unknown-linux-gnu (Debian/2.2)
407 
408 ## 0.9.2
409 
410  * Improve delta algorithm so that deltas are actually
411  delta-compressed, rather than faked.
412 
413 ## 0.9.1
414 
415  * Rename the library to `librsync'.
416 
417  * Portability fixes.
418 
419  * Include the popt library, and use it to build rdiff if the library
420  is not present on the host.
421 
422  * Add file(1) magic for rdiff.
423 
424  * Add more to the manual pages.
425 
426  * It's no longer necessary to call rs_buffers_init on a stream before
427  starting to use it: all the internal data is kept in the job, not
428  in the stream.
429 
430  * Rename rs_stream_t to rs_buffers_t, a more obvious name. Pass the
431  buffers to every rs_job_iter() call, rather than setting it at
432  startup. Similarly for all the _begin() functions.
433 
434  * rs_job_new also takes the initial state function.
435 
436  * Return RS_PARAM_ERROR when library is misused.
437 
438 ## 0.9.0
439 
440  * Redesign API to be more like zlib/bzlib.
441 
442  * Put all command-line functions into a single rdiff(1) program.
443 
444  * New magic number `rs6'
445 
446  * Change to using popt for command line parsing.
447 
448  * Use Doxygen for API documentation.
449 
450 ## 0.5.7
451 
452  * Changes stats string format.
453 
454  * Slightly improved test cases
455 
456 ## 0.5.6
457 
458  * Don't install debugging tools into /usr/local/bin; leave them in
459  the source directory.
460 
461  * Fix libhsync to build on (sgi-mips, IRIX64, gcc, GNU Make)
462 
463  * Include README.CVS in tarball
464 
465  * Back out of using libtool and shared libraries, as it is
466  unnecessary at this stage, complicates installation and slows down
467  compilation.
468 
469  * Use mapptr when reading data to decode, so that decoding should
470  have less latency and be more reliable.
471 
472  * Cope better on systems that are missing functions like snprintf.
473 
474 ## 0.5.5
475 
476  * Put genuine search encoding back into the nad algorithm, and
477  further clean up the nad code. Literals are now sent out using a
478  literal buffer integrated with the input mapptr so that data is not
479  copied. Checksums are still calculated from scratch each time
480  rather than by rolling -- this is very slow but simple.
481 
482  * Reshuffle test cases so that they use files generated by hsmapread,
483  rather than the source directory. This makes the tests quicker and
484  more reproducible, hopefully without losing coverage. Further
485  develop the test driver framework.
486 
487  * Add hsdumpsums debugging tool.
488 
489  * Hex strings (eg strong checksums) are broken up by underscores for
490  readability.
491 
492  * Stats now go to the log rather than stdout.
493 
494  * mapptr acts properly when we're skipping/rewinding to data already
495  present in the buffer -- it does a copy if required, but not
496  necessarily real IO.
497 
498 ## 0.5.4
499 
500  * Improved mapptr input code
501 
502  * Turn on more warnings if using gcc
503 
504  * More test cases
505 
506 ## 0.5.3
507 
508  * Improvements to mapptr to make it work better for network IO.
509 
510  * Debug trace code is compiled in unless turned off in ./configure
511  (although most programs will not write it out unless asked.)
512 
513  * Add libhsyncinfo program to show compiled-in settings and version.
514 
515  * Add test cases that run across localhost TCP sockets.
516 
517  * Improved build code; should now build easily from CVS through
518  autogen.sh.
519 
520  * Improved trace code.
521 
522  * Clean up to build on sparc-sun-solaris2.8, and in the process clean
523  up the handling of bytes vs chars, and of building without gcc
524 
525  * Reverse build scripts so that driver.sh calls the particular
526  script.
527 
528 ## 0.5.2
529 
530  * Use mapptr for input.
531 
532  * Implement a new structure for encoding in nad.c. It doesn't
533  encode at the moment, but it's much more maintainable.
534 
535  * More regression cases.
536 
537  * Clean up build process.
538 
539 ## 0.5.0
540 
541  * Rewrite hs_inbuf and hs_encode to make them simpler and more
542  reliable.
543 
544  * Test cases for input handling.
545 
546  * Use the map_ptr idea for input from both streams and files.
547 
548 ## 0.4.1
549 
550  * automake/autoconf now works cleanly when the build directory is
551  different to the source directory.
552 
553  * --enable-ccmalloc works again.
554 
555 ## 0.4.0
556 
557 * A much better regression suite.
558 
559 * CHECKSUM token includes the file's checksum up to the current
560  location, to aid in self-testing.
561 
562 * Various bug fixes, particularly to do with short IO returns.