Open Photography Forums  
HOME FORUMS NEWS FAQ SEARCH

Go Back   Open Photography Forums > Digital Darkroom > Image Processing and Workflow

Image Processing and Workflow RAW, DNG , TIFF and JPG. From Capture to Ready for Publish/Display. All software and techniques used within an image workflow, (except extensive retouching and repair or DAM).

Reply
 
Thread Tools Display Modes
  #1  
Old July 1st, 2006, 03:43 PM
Daniel_Hyams Daniel_Hyams is offline
New Member
 
Join Date: Jun 2006
Location: Chattanooga, TN
Posts: 17
Default Desire to write a new, free raw converter

This is just a feeler to see who would be interested in helping write a new raw converter, to fill the need of enthusiasts who need good raw conversion.

I've been mulling over it for a bit, and I realize that there is not a way for me to do this all myself, given the current state-of-the-art of tools like Bibble, Silkypix, and formerly RSE/P.

Initial rough requirements are:
#1 efficient photographer workflow for RAWs and JPGs
#2 rudimentary controls for exposure compensation, contrast, etc.
#3 background batch conversion
#4 pluggable color profiles that enable the use of 3rd party calibration
#5 initial RAW algorithm comes from dcraw, so there will be support for many cameras initially
#6 processing parameters saved in a standardizable XML sidecar file

I figure that would be more than enough to get started. I am a developer myself, and I was hoping that those enthusiasts with special expertise in areas such as color management and image manipulation might pipe up, and perhaps we can start working from there.

For cross-platform compatiblity, I thought that we might try the Qt API; this way, Windows, Mac, and Linux are all supported.

Last edited by Daniel_Hyams; July 1st, 2006 at 09:17 PM.
Reply With Quote
  #2  
Old July 2nd, 2006, 05:41 AM
Michael Tapes Michael Tapes is offline
OPF Administrator/Moderator
 
Join Date: Apr 2006
Posts: 243
Default

Hi Daniel,

I applaud your enthusiasm. From my perspective of having co-written an inexpensive RAW converter (YarcPlus), it is a labor of love. Given the camera purchases being primarily Canon and Nikon (see the OpenRAW survey), and that DPP (free) is always improving, I am not sure the purpose of the free converter. I would say that supplying DCRAW in some modifued and legal forms might be the way to go, but I think that there are better ways to serve the RAW community. But then again, if you are up for it, then go for it!

Good luck. I do not think I can help, but certainly call on me for comment, information, or suggestions...

My best...
__________________
Michael Tapes
Owner: Michael Tapes Design
Creator: WhiBal - Instant JPEG from Raw - LensAlign
Reply With Quote
  #3  
Old July 2nd, 2006, 06:24 AM
Daniel_Hyams Daniel_Hyams is offline
New Member
 
Join Date: Jun 2006
Location: Chattanooga, TN
Posts: 17
Default

I appreciate the sentiment, Michael; I know that it can definitely be a labor of love, as I have written one other software package, and it spanned over years.

My hope is that I can gather some expertise in certain areas, because I just don't have the background to cover all of the areas that need to be covered for a good raw converter.
Reply With Quote
  #4  
Old July 2nd, 2006, 07:58 AM
Michael Tapes Michael Tapes is offline
OPF Administrator/Moderator
 
Join Date: Apr 2006
Posts: 243
Default

A serious suggestion. Another way to improve the state of the art is to feed back to Adobe during their Lightroom preview period. I can tell you that they take all input very seriously. I know it is not out for Windows yet, but should be out in 6-8 weeks...my best guess...

Just a thought...
__________________
Michael Tapes
Owner: Michael Tapes Design
Creator: WhiBal - Instant JPEG from Raw - LensAlign
Reply With Quote
  #5  
Old July 2nd, 2006, 10:56 AM
Daniel_Hyams Daniel_Hyams is offline
New Member
 
Join Date: Jun 2006
Location: Chattanooga, TN
Posts: 17
Default

Yep, I plan to do that; I'm already signed up for the beta. Lightroom will probably be very, very good, but the wondering for me is at what price it will be offered. If it is around $100 or even $150, I'll be interested. Otherwise, I feel like I have nowhere else to go for Raw conversion....each of the other packages suffer from warts I would rather not put up with.

There was obviously a market for RawShooter due to the fast workflow and good-to-excellent raw conversion (with CE, not without) among non-pro enthusiasts, and I just hate to see that hole remain now that RSP has been yanked from the pool.
Reply With Quote
  #6  
Old July 2nd, 2006, 11:43 AM
Ray West Ray West is offline
Senior Member
 
Join Date: Jun 2006
Location: UK - Somerset
Posts: 1,703
Default

Hi Daniel,

I had a quick look at the dcdraw site, but could not find a list of cameras supported, or does it work in 'some other way' ? What happens to your software, if dcdraw fizzles out?

Was it your intention to produce jpegs or tiffs, or whatever from the initial jpeg or raw file, and how much detail are you storing in the xml file? or is this stuff yet to be decided....?

I'm woindering if an alternative method may be to take the tiff files, produced by many camera manu's free software, and then batch process them into something better. I guess that is possible, provided they don't blow the highlights, whatever. In reality, I think the camera manufacturer will know more about the requirements for their raw processing, compared to a third party trying to reverse engineer their encodings.

I think I could generate some interesting visual effects - most of thenm being unintentional, I expect

Best wishes,

Ray
Reply With Quote
  #7  
Old July 2nd, 2006, 02:12 PM
Daniel_Hyams Daniel_Hyams is offline
New Member
 
Join Date: Jun 2006
Location: Chattanooga, TN
Posts: 17
Default

Good questions Ray,

The list of cameras supported is at the bottom of http://www.cybercom.net/~dcoffin/dcraw/ , and the list is very impressive. My thought was to use dcraw initially, just to get started, and then migrate toward a unique demosaicing algorithm, if needed at all. If dcraw goes by the wayside, we would be no worse off then if we never used it in the first place.

The intention is to produces tiffs and/or jpgs, as most raw converters do. Of course, there is nothing stopping us from producing other filetypes such as pngs. In the beginning, in the interest of keeping things simple, we might just do jpegs.

As far as the suggestion to use tiffs, by that time the camera manufacturer has imposed their interpretation of color, detail, etc. etc. Also, the user would have had to do the processing once, and the objective of this software would be to make things as easy for the user as possible.

It is important to distinguish between the *initial* list of features to get started, which should be kept small, and the list of features that we might want to work to. What I have listed here is an initial list of features, intended to be easy enough to attain, but rich enough to generate usage and interest from the general photographic community. In the end, I can see the integration things like distortion correction, cropping, straightening, curves and levels, and etc.
Reply With Quote
  #8  
Old July 2nd, 2006, 02:15 PM
Daniel_Hyams Daniel_Hyams is offline
New Member
 
Join Date: Jun 2006
Location: Chattanooga, TN
Posts: 17
Default

And here's the camera support list for dcraw:

Adobe Digital Negative (DNG)
AVT F-145C
AVT F-201C
AVT F-510C
AVT F-810C
Canon PowerShot 600
Canon PowerShot A5
Canon PowerShot A5 Zoom
Canon PowerShot A50
Canon PowerShot Pro70
Canon PowerShot Pro90 IS
Canon PowerShot G1
Canon PowerShot G2
Canon PowerShot G3
Canon PowerShot G5
Canon PowerShot G6
Canon PowerShot S30
Canon PowerShot S40
Canon PowerShot S45
Canon PowerShot S50
Canon PowerShot S60
Canon PowerShot S70
Canon PowerShot Pro1
Canon EOS D30
Canon EOS D60
Canon EOS 5D
Canon EOS 10D
Canon EOS 20D
Canon EOS 30D
Canon EOS 300D / Digital Rebel / Kiss Digital
Canon EOS 350D / Digital Rebel XT / Kiss Digital N
Canon EOS D2000C
Canon EOS-1D
Canon EOS-1DS
Canon EOS-1D Mark II
Canon EOS-1D Mark II N
Canon EOS-1Ds Mark II
Casio QV-2000UX
Casio QV-3000EX
Casio QV-3500EX
Casio QV-4000
Casio QV-5700
Casio QV-R51
Casio QV-R61
Casio EX-S100
Casio EX-Z50
Casio EX-Z55
Casio Exlim Pro 505
Casio Exlim Pro 600
Casio Exlim Pro 700
Contax N Digital
Creative PC-CAM 600
Epson R-D1
Foculus 531C
Fuji FinePix E550
Fuji FinePix E900
Fuji FinePix F700
Fuji FinePix F710
Fuji FinePix F800
Fuji FinePix F810
Fuji FinePix S2Pro
Fuji FinePix S3Pro
Fuji FinePix S20Pro
Fuji FinePix S5000
Fuji FinePix S5100/S5500
Fuji FinePix S5200/S5600
Fuji FinePix S7000
Fuji FinePix S9000/S9500
Imacon Ixpress 16-megapixel
Imacon Ixpress 22-megapixel
ISG 2020x1520
Kodak DC20 (see Oliver Hartman's page)
Kodak DC25 (see Jun-ichiro Itoh's page)
Kodak DC40
Kodak DC50
Kodak DC120 (also try kdc2tiff)
Kodak DCS315C
Kodak DCS330C
Kodak DCS420
Kodak DCS460
Kodak DCS460A
Kodak DCS520C
Kodak DCS560C
Kodak DCS620C
Kodak DCS620X
Kodak DCS660C
Kodak DCS660M
Kodak DCS720X
Kodak DCS760C
Kodak DCS760M
Kodak EOSDCS1
Kodak EOSDCS3B
Kodak NC2000F
Kodak ProBack
Kodak PB645C
Kodak PB645H
Kodak PB645M
Kodak DCS Pro 14n
Kodak DCS Pro 14nx
Kodak DCS Pro SLR/c
Kodak DCS Pro SLR/n
Kodak P850
Kodak P880
Kodak KAI-0340
Konica KD-400Z
Konica KD-510Z
Leaf Aptus 17
Leaf Aptus 22
Leaf Aptus 65
Leaf Aptus 75
Leaf Valeo 6
Leaf Valeo 11
Leaf Valeo 17
Leaf Valeo 22
Leaf Volare
Leica Digilux 2
Leica D-Lux 2
Logitech Fotoman Pixtura
Micron 2010
Minolta RD175
Minolta DiMAGE 5
Minolta DiMAGE 7
Minolta DiMAGE 7i
Minolta DiMAGE 7Hi
Minolta DiMAGE A1
Minolta DiMAGE A2
Minolta DiMAGE A200
Minolta DiMAGE G400
Minolta DiMAGE G500
Minolta DiMAGE G530
Minolta DiMAGE G600
Minolta DiMAGE Z2
Minolta Alpha/Dynax/Maxxum 5D
Minolta Alpha/Dynax/Maxxum 7D
Nikon D1
Nikon D1H
Nikon D1X
Nikon D100
Nikon D2H
Nikon D2Hs
Nikon D2X
Nikon D200
Nikon D50
Nikon D70
Nikon D70s
Nikon E700 ("DIAG RAW" hack)
Nikon E800 ("DIAG RAW" hack)
Nikon E880 ("DIAG RAW" hack)
Nikon E900 ("DIAG RAW" hack)
Nikon E950 ("DIAG RAW" hack)
Nikon E990 ("DIAG RAW" hack)
Nikon E995 ("DIAG RAW" hack)
Nikon E2100 ("DIAG RAW" hack)
Nikon E2500 ("DIAG RAW" hack)
Nikon E3700 ("DIAG RAW" hack)
Nikon E4300 ("DIAG RAW" hack)
Nikon E4500 ("DIAG RAW" hack)
Nikon E5000
Nikon E5400
Nikon E5700
Nikon E8400
Nikon E8700
Nikon E8800
Olympus C3030Z
Olympus C5050Z
Olympus C5060WZ
Olympus C7070WZ
Olympus C70Z,C7000Z
Olympus C740UZ
Olympus C770UZ
Olympus C8080WZ
Olympus E-1
Olympus E-10
Olympus E-20
Olympus E-300
Olympus E-330
Olympus E-500
Olympus SP350
Olympus SP500UZ
Panasonic DMC-FZ30
Panasonic DMC-LC1
Panasonic DMC-LX1
Pentax *ist D
Pentax *ist DL
Pentax *ist DS
Pentax Optio S
Pentax Optio S4
Pentax Optio 33WR
Phase One LightPhase
Phase One H 10
Phase One H 20
Phase One H 25
Phase One P 20
Phase One P 25
Phase One P 30
Phase One P 45
Pixelink A782
Polaroid x530
Rollei d530flex
RoverShot 3320af
Sarnoff 4096x5440
Sigma SD9
Sigma SD10
Sinar 3072x2048
Sinar 4080x4080
Sinar STI format
SMaL Ultra-Pocket 3
SMaL Ultra-Pocket 4
SMaL Ultra-Pocket 5
Sony DSC-F828
Sony DSC-R1
Sony DSC-V3
Sony DSLR-A100
Sony XCD-SX910CR
STV680 VGA
Reply With Quote
  #9  
Old July 10th, 2006, 03:59 AM
Juergen Koslowski Juergen Koslowski is offline
New Member
 
Join Date: Jun 2006
Posts: 18
Default

Daniel,

I have much sympathy with your project in principle. But which operating system are you aiming for? Presently, users of Linux and/or other free operating systems are very limited in their options when it comes to raw processing. I only fire up Windows XP for image processing, also due to the lack of 16bit color depth support of the GIMP (a severe mistake of that project, in my view; the situation with CinePaint seems to be unsatisfactory as well, despite 16 bit support).

In case you are considering Linux, you will also have to deal with the state of color calibration there, which may not be as highly evolved as in the Windows and/or Mac world. It would be very nice, if this could be changed.

Presently, I'm mostly using Silkypix, after being unable to get similar colors in RSP (now sadly defunct), even with the Color Engine. Silkypix provides a large number of options for raw development, which minimizes the need for postprocessing in a separate program. One additional feature, I would like to see, would be a healing tool. Presently, besides keeping the raw files, I am saving processed 16 bit files (in lossless JPEG2000 format) where I have removed the dust spots, which invariably occur. Resizing and sharpening is then performed on those preprocessed files, since I do not want to remove the dust spots again, it is way to easy to miss one.

JPEG2000 support is not really well developed. Even when using Image Magick, it seems to be difficult to embed color profiles. However, lossless 16bit JPEG2000 files on average seem to be slightly smaller that corresponding png files (and the programs that reduce png files in size only seem to work on 8 bit versions and take an enourmous amount of time). Unless the issue of a healing brush can be addressed at the raw level, an efficient file format for lossless 16 bit (or higher) image files with the ability to add meta data (color profiles and EXIF data) for archival purposes would be useful. But in order to address storage requirements, a meta file with instructions how to develop a raw file (including healing brush applications) would still be preferable.

Compared to roughly 13MB - 14MB for a raw file from my EOS 5D, a 16 bit tiff-file consume 72MB, which can be reduced to approximately 49MB in case of lossless JPEG2000 (and roughly 52MB in case of png).

I do not know, whether Lightroom addresses these issues.

-- Jürgen (not a programmer, just a mathematician ;-)

Last edited by Juergen Koslowski; July 10th, 2006 at 04:21 AM.
Reply With Quote
  #10  
Old July 11th, 2006, 09:08 AM
Daniel_Hyams Daniel_Hyams is offline
New Member
 
Join Date: Jun 2006
Location: Chattanooga, TN
Posts: 17
Default

Quote:
Originally Posted by Juergen Koslowski
Daniel,

I have much sympathy with your project in principle. But which operating system are you aiming for? Presently, users of Linux and/or other free operating systems are very limited in their options when it comes to raw processing. I only fire up Windows XP for image processing, also due to the lack of 16bit color depth support of the GIMP (a severe mistake of that project, in my view; the situation with CinePaint seems to be unsatisfactory as well, despite 16 bit support).

In case you are considering Linux, you will also have to deal with the state of color calibration there, which may not be as highly evolved as in the Windows and/or Mac world. It would be very nice, if this could be changed.
Anything that I write will be cross platform, meaning Windows, Mac, and Linux. The color management in Linux might be a problem, but I'm hoping with the LittleCMS library, I might be able to accomplish it.

Quote:
Presently, I'm mostly using Silkypix, after being unable to get similar colors in RSP (now sadly defunct), even with the Color Engine. Silkypix provides a large number of options for raw development, which minimizes the need for postprocessing in a separate program. One additional feature, I would like to see, would be a healing tool.
This is definitely one of the features I want also, so it will be in there if this comes to fruition. I had envisioned a simple stamp tool, where one selects the radius of the brush and then does a "source click" and "destination click" in order to copy pixels from source to destination. Not as fancy as Photoshop's stamp tool, but certainly good enough to fix problems such as dust spots and the like. Another advantage of this simple interface is that the actions can simply be recorded in the sidecar file, and are therefore applicable no matter what future development the image undergoes.

Quote:
JPEG2000 support is not really well developed.
This is not something I've given much thought to, but I will take a look into it...

I've not really received much interest in making this an open source project, so I've been forging ahead on my own; I think that progress has been decent thus far. But instead of it being an open source project, it would probably be a product that I would charge for in the end.
Reply With Quote
  #11  
Old July 12th, 2006, 07:42 PM
Josh Liechty Josh Liechty is offline
Member
 
Join Date: May 2006
Location: Indiana, USA
Posts: 51
Default

If it were an open source project, I would have liked to gotten involved, except that my knowledge of C/C++ is very rudimentary at best, so I'd be more of a burden than a help. However, if it's a commercial product, then unless it were to have very compelling advantages and were interoperable with an Adobe Bridge/ACR + DNG workflow, I wouldn't be interested.
__________________
Josh Liechty
Reply With Quote
  #12  
Old July 17th, 2006, 04:28 AM
John Burkus John Burkus is offline
New Member
 
Join Date: Jun 2006
Location: Oakville, ON, Canada
Posts: 11
Default

I have been following this thread for a few days and I wonder why anyone would want to take on such a thankless project that will lead to an incredible investment in future time and resources.

There is no way that a free RAW converter will compete with the likes of reasonably priced RAW converters such as Silkypix, which I use, Bibble, ACDC Pro, Helicon Filter, or, say, Picture Window Pro.

If you read any of the forums associated with their sites what you get is an never ending litany of needs, complaints, bugs, crashes and lists of cameras that are not yet compatible with the program.
__________________
John

Light and Shadow

"Great meal. What pots did you use?"
Reply With Quote
  #13  
Old July 17th, 2006, 04:41 PM
Daniel Harrison Daniel Harrison is offline
Member
 
Join Date: Jun 2006
Location: Melbourne, Australia
Posts: 236
Default

You all might find this interesting, may even want to get intouch with the guy. Even works with linux!

http://forums.dpreview.com/forums/re...ssage=19187667
__________________
-------------------
Daniel Harrison
Reply With Quote
  #14  
Old August 1st, 2006, 01:43 PM
Peter Ruevski Peter Ruevski is offline
Member
 
Join Date: Jul 2006
Posts: 39
Default

Quote:
Originally Posted by Daniel_Hyams
This is just a feeler to see who would be interested in helping write a new raw converter, to fill the need of enthusiasts who need good raw conversion.

I've been mulling over it for a bit, and I realize that there is not a way for me to do this all myself, given the current state-of-the-art of tools like Bibble, Silkypix, and formerly RSE/P.
Hi Daniel,

As a programmer I have been considering raw conversion tools for some time myself. I have so many thoughts about what I want in such a tool how it should work, what the architecture should look like and so on.

Presumably when you say "write" you mean open source? If so you should definitely look at UFRaw http://ufraw.sourceforge.net/
It is both a stand alone program and a GIMP plugin, portable (based on the GTK GUI toolkit) and is getting better all the time. It even has some color management capabilities based on LittleCMS. So it is definitely worth looking at and maybe contributing code/ideas.

Best regards,
Peter

P.S. I am not affiliated with UFRaw in any way and have not - so far - contributed to the project so there is no "commercial" :-))) interest.
Reply With Quote
Reply

Bookmarks

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are Off
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 05:52 AM.


Posting images or text grants license to OPF, yet of such remain with its creator. Still, all assembled discussion 2006-2014 Asher Kelman (all rights reserved) Posts with new theme or unusual image might be moved/copied to a new thread!