Rawshack - Raw Image Analyzer

This is the official homepage of Rawshack, a raw image analyzer utility. Rawshack reports a variety of satsitical information about a raw image file, including pixel count data that can be used to generate your own histogram using external programs like Microsoft Excel. Rawshack is an open source application, licensed under GPL v3 and uses the LibRaw Library. It supports most bayer-filter sensor cameras.


Date Ver Change List Source Code LibRaw Source My LibRaw changes Windows Binary (32-bit) Windows Binary (64-bit) Mac OSX Binary (64-bit) Linux Binary (64-bit)
06/14/2017 1.06 changelist source (zip) LibRaw-0.18.2.zip LibRaw Patch File rawshack32.exe rawshack.exe rawshack rawshack
06/09/2017 1.05 changelist source (zip) LibRaw-0.18.2.zip LibRaw Patch File rawshack.exe   rawshack rawshack
06/05/2017 1.04 changelist source (zip) LibRaw-0.18.2.zip LibRaw Patch File rawshack.exe   rawshack rawshack
01/22/2014 1.03 changelist source (zip) LibRaw-0.16.0.zip LibRaw Patch File rawshack.exe   N/A N/A
04/19/2001 1.02 changelist source (zip) LibRaw-0.13.4.zip LibRaw Patch File rawshack.exe   N/A N/A
04/18/2011 1.01 changelist source (zip) LibRaw-0.13.4.zip LibRaw Patch File rawshack.exe   N/A N/A
04/16/2011 1.00 Initial Release rawshack.cpp LibRaw-0.13.4.zip LibRaw Patch File rawshack.exe   N/A N/A


rawshack [optional args] filename(s)

You can specify one or more filenames to be processed. You can also use a wildcard specifier, which directs Rawshack to process every file whose name matching the specification. For example, "c:\mypics\*.cr2" will process all files in the c:\mypics directory that have a .cr2 extension. You can use multiplie wildcard specifications, like "c:\mypics\*.cr2 c:\mypics\*.nef", to process all .cr2 and .nef files in the c:\mypics directory.

Optional arguments:

This help display

Performs black-level bias subtraction of image data. Some cameras, like Canons, apply a black offset to the analog signal data prior to quantization through the ADC (Analog-to-Digital converter). This allows for the recreation of signal data at the very bottom of the noise floor, which is important for certain applications like astrophotography. If --blacksubtraction is not specified, the original ADU (Analog-to-Digital Unit) values will be analyzed, which includes the analog bias. If --blacksubtraction is specified, the ADU values above the black level will be reduced by the black level, and all ADU values equal to or below the black level will be reset to zero.

--clippinglevels=default | channelmax | bitdepth | r,g,b,g2
Set raw clipping levels. The clipping levels determine which pixel values are interpreted by Rawshack as "clipped" or fully saturated. Having the correct clipping levels is essential for the proper reporting of Rawshack's clipping statistics. If the --clippinglevels option is not specified, Rawshack will use the clipping levels it has defined for the camera in its internal database of cameras. If there is no database entry for that camera, or if the channel maximums for the specific raw image are higher than the levels specified in the database entry then Rawshack will use the channel maximums. "Channel maximums" simply means the highest pixel value that was found for each of the 4 bayer color channels.

You can direct Rawshack to always ignore the internal database by using "channelmax" (always use the channel maximums for clipping), "bitdepth" (use the maximum value that can be represented by the bitdepth of the raw as the clipping value, provided the bitdepth was available for the raw). You can also directly specify the absolute clipping values for the channels. For example, "--clippinglevels=15580,15580,15575,15400" will set the clipping level of the red and green channels to 15580, the blue channel to 15575, and the the green_2 channel to 15400.

Set dimensions of image subset to analyze. By default Rawshack will analyze the entire contents of the image, net of any camera-specific border/masked areas. This option lets you direct Rawshack to analyze only a subset of the image area.

By default, Rawshack generates the analyzed statistics of each raw into a file named <rawfilename>.txt. For example, if your raw was named dsc_0092.nef, the statistics file will be named dsc_0092.txt. Specifying --nostatsfile disables the generation of this file, directing the output to stdout instead (console screen).

Rawshack can optionally generate a separate output file that contains the count of pixel values at each ADU. This data can then be used via an external program like Microsoft Excel to generate histograms on a per-channel or channel-combined basis. The name of the generated file is <rawfilename>.hist.txt. The output is a comma-delimited ASCII file with a legend on the first row, followed by the pixel values for each ADU. Here is a sample output:

    ADU, Red, Green, Blue, Green_2

The first row the legend. Each subsequent row has the ADU in the first column, which starts from 0 and ends at either the higher of all channel maximums or configured clipping levels, whicheveri s greater. The next 4 columns describe the count of pixels found for that ADU in each color channel.

By default, Rawshack stores the output files (statistics, histogram data, etc...) in the current working directory. Specify the --outputinsrcdir option if you'd like the output files to be placed in the same directory as the raw file.

--sonycompdata=levels (V1.03)
Generates a comma-delimited ASCII file describing all 16x2 pixel blocks that are encoded with delta tonal lossy compression levels matching the levels specified for this option. There are 5 compression levels and any combintation of them can be specified in the levels option. Level 0 represents pixels encoded with lossless compression, Level 4 represents pixels with the most lossy compression (and varying tiers of lossy compression in between). For example, to report only pixels with the highest level of lossy compression (Level 4), use "--sonycompdata=4". To report pixels using the two most lossy compression levels use "--sonycompdata=3,4". If you'd like to report pixels for all levels (every pixel in raw), use "--sonycompdata=0,1,2,3,4". The default if no levels are specified is 3,4. See my dpreview thread here for more details on Sony's lossy compression and how this option can be used to detect image artifacts that may result from it.

The name of the generated file is <rawfilename>.sonycomp.csv and is formatted as follows:

First Line:

    Sony Camera Model, LibRaw image width, LibRaw image height, Hex value describing --sonycompdata levels value (encoded bits)

All Subsequent lines:

    Pixel X-coordinate, Pixel Y-coordinate, Compression Level of Pixel

Sample Data:


A PNG of the resulting CSV file can be generated using my createsonypng PHP script (download here - save it as createsonypng.php from your browser). View the comments at the top of the script for instructions on how to install and configure PHP. The script will color-code each pixel based on its compressoin level. Level 4 = White, Level 3 = Red, Level 2 = Yellow, Level 1 = Blue, Level 0 = Green). Here is a sample PNG generated from the script.

--columngroupstats=#cols,... (V1.05)
--rowgroupstats=#rows,... (V1.05)
Generates a seperate set of stddev stats (mean, stddev, and COV) by binning pixel stats into groups of the specified number of columns (--columngroupstats) or rows (--rowgroupstats). For example, if the raw is 6000x4000 and you specify --columngroupstats=100, rawshack will geneate stats for 60 seperate column bins, each 100 pixels wide (pixels in columns 0..99, pixels columns 100..199, etc...). The resulting stats for each specified column or row group will be written into 4 files, named <rawfilename>.cols-xxx-yyy.txt (--columngroupstats) or <rawfilename>.rows-xxx-yyy.txt (--rowgroupstats), where 'xxx' is the nmber of pixels in the group and 'yyy' is the color channel (Red, Green, Blue, and Green_2). You can specify multiple column and row groups to be generated at the same time (ex: --rowgroupstats=64,32,16,8). Here is a sample of the output for one of the group stat files:


--recurse (V1.06)
Recurses through subdirectories when performing a wildcard search. Subdirectories will only be searched for file arguments that contain a wildcard, such as *.NEF. When using this option on OSX and Linux, put quotes around the path, otherwise the shell will perform a wildcard replacement without passing the wildcards to rawshack. Example: "/MyVol/MyPics/*.arw" --recurse

--maxthreads (V1.06)
The maximum number of image processing threads. Default is the number of processor cores detected on the system (default is capped at 4 on the 32-bit Windows executable due to the 3GB memory constraint). Be sure you have sufficient physical memory to support the thread count you specify otherwise performance will degrade from OS virtual memory paging file thrashing - each raw file can require hundred megabytes of memory, so the total memory consumed will be #threads x #files.

--maxconcurrentreads (V1.06)
The maximum number of files that will be read simultaneously across image processing threads. The default is two, which prevents performance degradation from disk thrashing on spinning disk drives.

Sample Output

                 File: C:\develop\testraw\D7000_ISO100.NEF
               Camera: NIKON D7000
      Exposure/Params: ISO 100 f/8 1/500s 24mm
            Bit Depth: 14-bit
     Image dimensions: 4948x3280 (pixel count = 16,229,440)
  Analyzed image rect: 0000x0000 to 4948x3280 (pixel count = 16,229,440)
  Clipping levels src: Channel Maximums
      Clipping levels: Red=16,383; Green=15,779; Blue=16,383; Green_2=15,779
          Black point: 0

Min pixel value:
      Red: pixVal=      3, count =           1, first coord: 4094x0018
    Green: pixVal=     12, count =           1, first coord: 1175x2882
     Blue: pixVal=      0, count =           1, first coord: 0001x1525
  Green_2: pixVal=     13, count =           2, first coord: 1178x2883

Min non-zero pixel value:
      Red: pixVal=      3, count =           1, first coord: 4094x0018
    Green: pixVal=     12, count =           1, first coord: 1175x2882
     Blue: pixVal=      5, count =           1, first coord: 0763x2439
  Green_2: pixVal=     13, count =           2, first coord: 1178x2883

Max pixel value:
      Red: pixVal= 16,383, count =       2,694, first coord: 4258x0420
    Green: pixVal= 15,779, count =           2, first coord: 1151x0018
     Blue: pixVal= 16,383, count =       3,480, first coord: 1151x0019
  Green_2: pixVal= 15,779, count =           1, first coord: 1152x0017

Max non-clipped pixel value:
      Red: pixVal= 16,381, count =           1, first coord: 2536x1646
    Green: pixVal= 15,778, count =       4,540, first coord: 4247x0414
     Blue: pixVal= 16,379, count =           1, first coord: 2537x1627
  Green_2: pixVal= 15,778, count =       4,543, first coord: 2510x0167

Clipped pixel counts (individual channels):
      Red: count =       2,694, first coord: 4258x0420
    Green: count =           2, first coord: 1151x0018
     Blue: count =       3,480, first coord: 1151x0019
  Green_2: count =           1, first coord: 1152x0017

Clipped pixel counts (channel combinations) for 2x2 bayer pixel GRBG matrix:
    None:  4053489,        R:      389,        G:        1,      R+G:        0
       B:     1174,      R+B:     2305,      G+B:        1,    R+G+B:        0
      G2:        1,     R+G2:        0,     G+G2:        0,   R+G+G2:        0
    B+G2:        0,   R+B+G2:        0,   G+B+G2:        0, R+G+B+G2:        0

Lowest pixel value bit set:
      Red: bit= 0, count =   2,037,231, first coord: 0000x0000
    Green: bit= 0, count =   2,021,157, first coord: 0001x0000
     Blue: bit= 0, count =   2,037,761, first coord: 0001x0001
  Green_2: bit= 0, count =   2,020,726, first coord: 0004x0001

Highest non-clipped pixel value bit set:
      Red: bit=13, count =       1,680, first coord: 1150x0016
    Green: bit=13, count =       9,570, first coord: 1151x0016
     Blue: bit=13, count =       2,827, first coord: 1151x0017
  Green_2: bit=13, count =       9,525, first coord: 1150x0017

Pixel average and standard deviation:
      Red: Mean=  401.93, StdDev=  713.17, COV=  177.44%, Unique values=  7,481
    Green: Mean=  768.60, StdDev= 1179.69, COV=  153.49%, Unique values= 11,461
     Blue: Mean=  550.96, StdDev=  913.96, COV=  165.89%, Unique values=  8,861
  Green_2: Mean=  770.56, StdDev= 1180.79, COV=  153.24%, Unique values= 11,484

Pixel counts for each raw stop:
    Stop #/Range         Red        Green         Blue      Green_2
 ---------------  ----------   ----------   ----------   ----------
 0 [00000-00001]           0            0            1            0
 1 [00002-00003]           1            0            0            0
 2 [00004-00007]          41            0            1            0
 3 [00008-00015]       3,548           13          451            5
 4 [00016-00031]      44,801        5,817       24,940        5,491
 5 [00032-00063]     182,871       64,706      147,800       63,880
 6 [00064-00127]     890,952      231,445      615,279      229,885
 7 [00128-00255]   1,616,572      965,524    1,109,004      960,707
 8 [00256-00511]     531,739    1,418,037    1,181,971    1,419,200
 9 [00512-01023]     362,200      643,914      437,993      648,758
10 [01024-02047]     326,237      320,510      293,485      321,209
11 [02048-04095]      89,345      308,370      221,420      308,597
12 [04096-08191]       4,679       89,452       18,708       90,102
13 [08192-16383]       4,374        9,572        6,307        9,526
14 [16384-32767]           0            0            0            0
15 [32768-65535]           0            0            0            0

GPL v3 License

    Rawshack - Raw Image Analyzer
    Copyright (C) 2011;  adam@testcams.com

    This program is free software: you can redistribute it and/or modify
    it under the terms of the GNU General Public License as published by
    the Free Software Foundation, either version 3 of the License, or
    (at your option) any later version.

    This program is distributed in the hope that it will be useful,
    but WITHOUT ANY WARRANTY; without even the implied warranty of
    GNU General Public License for more details.

    You should have received a copy of the GNU General Public License
    along with this program.  If not, see <http://www.gnu.org/licenses/>.


If you have a feature suggestion or bug to report (which is likely, considering it's brand new!), please contact me at adam@testcams.com