I just found a long needed tool, Converseen (http://converseen.sourceforge.net) which is a batch photo format converter and resizer.
It does what other apps do (e.g. ImageMagick) but is more point and click, which suits me as I cannot remember the steps to take in ImageMagick between each photo session.
Since it took a few hours to process all thousands of photos I guess it could be used for benchmarking to.
Just a thought.
There's already a graphics-magick test profile that does benchmarks of image resizing and other types of image manipulation.