Testing an application like ClipWrap can be complicated. There are thousands of combinations of cameras, computers and editing applications that need to be taken into account.
Throughout the ClipWrap development process, we do a mix of automated and manual testing. We have a library of hundreds of test clips, in a mix of formats, which we feed through ClipWrap to verify that we haven’t introduced issues as we add features and improve performance. In fact, our list of supported cameras is fed by the same database that we use for testing – when you see an entry in that list, it means we’re actively testing with files from that camera.
Before we do a release, we’ll run our test suite across a variety of computers – from the newest Macbook Pros and Mac Pros running Lion, to older PowerMac G5s running Tiger or Leopard, and a mix of platforms in between. For every test file, we’ll generate an output in each of the 12 formats ClipWrap supports. And we’ll do that both with and without Perian installed, and with and without audio passthru. That’s many thousands of outputs! Thankfully, because of ClipWrap’s powerful command line interface, we can automate much of that testing.
Of course, we do lots of manual testing as well – making sure we haven’t overlooked something obvious which the automated testing might not catch.
Even with all of that testing, problems can still arise. The biggest challenge we face are is that each month, new cameras hit the market. Many of these cameras are specific to a particular region of the globe, or contain a custom firmware for a particular market. This is where you come in – if you’re a ClipWrap user and obtain a new camera, we’d love it if you’d share some sample files with us. Just shoot us an email and we’ll give you access to our file sharing system. The more details you can provide – make, model, settings – the better we can refine out testing – and the better the product will be for everyone.