I was pretty mean to Colin in that post. In the intervening year I've come to respect Colin's practical experience a lot more than I did before, when I thought he was hopelessly academic. I'd like to make it clear that Colin doesn't look incompetant (actually, he seems to be earning accolades from the Twitterverse for how he wrote it up).
But yeah, I hope it's obvious that I see this as a very strong vindication for my argument that generalist devs shouldn't build crypto. At all, ever. Use TLS for data in motion; use PGP for data at rest. Systems much bigger and heavier than yours have gotten away with this.
Colin is wildly competent, but I was always uncomfortable with how much he was willing to do his own work without review. He is probably one of the best people in the world to do this work, but even the best person in the world will occasionally make mistakes, as happened here.
> Expert devs shouldn't build crypto without review.
I was surprised that Colin's solution is to personally re-review his code. Good writers know--don't rely on yourself for proofreading. Usually the mental lapse that caused the problem will manifest itself during your review as well.
In my experience, I'm very good at proofreading my own writing/code, as long as I wait long enough that I've forgotten it. In this case, I was looking at code which I wrote over a year ago.
But please, go ahead and give the code another read. :-)
I sense a great justification for an alcohol budget.
Edit: Totally willing to continue burning karma on this comment if the HN community continues to decide vote it down. I've tried reviewing my own code in a different state of intoxication than when I wrote it and I'm not joking that it can help. I'm still trying to pull resources together for a study on the benefit of different mindframes for peer review. We haven't tried alcohol yet, but frankly it wouldn't be a half bad idea if we could get anyone not to laugh too loudly at the proposal.
"they are wont to deliberate when
drinking hard about the most important of their affairs, and whatsoever
conclusion has pleased them in their deliberation, this on the next day,
when they are sober, the master of the house in which they happen to be
when they deliberate lays before them for discussion: and if it pleases them when they are sober also, they adopt it, but if it does not please them, they let it go: and that on which they have had the first deliberation when they are sober, they consider again when they are drinking."
I also agree with you in general, that checking things in different mental states is a good practice. With alcohol, I suspect the benefit is outweighed by the difficulty of spotting bugs when drunk -- but who knows?
Well, I am actually a bit more curious about the difference between coding drunk and checking sober vs. coding sober, checking sober... But I suppose checking drunk would be amusing too.
Cute idea, but I don't drink (for medical reasons).
I suppose I could try reviewing code in both caffeinated and decaffeinated states, but being decaffeinated gives me enough of a headache that I don't think I'd be much use that way.
There's some sort of analogy in here about how alcohol 'loosens you up' could be related to getting in the proper state of mind to 'code fearlessly,' but I can't seem to find it.
How do you assess your effectiveness at proofreading your own code?
I know that I find more mistakes in my code when time reveals the code as it is rather than as it was intended. But I can't say this makes me good enough at proofreading myself. What about the code I've conceived and written in ignorance?
> Use TLS for data in motion; use PGP for data at rest.
That's useful advice, if you need and _want_ the guarantees given by TLS or PGP. If you have other needs then a look at, say, off-the-record messaging may be useful.
I think it's a bad idea to recommend the OTR protocol to people looking for a simple encrypted transport (or simple encrypted record storage). How do you judge whether the guarantees TLS offers are "needed" or not?
That judgment is partly outside the more mechanical parts of cryptography. You have to see what your application domain demands.
OTR is just the first example I could think of, that gives different guarantees than most normal cryptosystems. I don't particularly recommend it for anything apart from instant messaging. And I wouldn't recommend implementing your own.
If I speak to you in private (and we know each other), you can be sure you are speaking to me, but you won't be able to proof to any third party anything I said. OTR can give you something like that. PGP can't.
For most application you will be well served with PGP or TLS. But be aware of what baggage they bring. For some areas losing deniability via PGP can be worse than plain text.
This is a counterfeit argument. PGP loses "deniability" (and "forward secrecy") if by PGP you mean "the PGP user interface". But if what you mean is simply "the PGP cryptosystem" and "the PGP message format of packets and bulk encryption and signatures", then you can grant your system most any property OTR gives you.
This is a moot point, because most systems would never care enough to intricately position all their features just-so to compose OTR-like features out of PGP primitives. What they need is to be able to encrypt anything without implementing trivially exploitable crypto vulnerabilities that were discovered and solved decades ago.
This is a textbook case of everyone's good being strangled by someone's opinion of the perfect.
I don't disagree with you. And OTR is just one example, and may even be a straw-man by now. Just be aware that there are other valid choices for cryptosystems, while you still don't have to roll your own.
Just curious:
For a similar system to tarsnap where local data is encrypted via pgp and stored on a remote system, what extra benefit is there to use TLS for the data transfer/data i motion?
No one is qualified to implement crypto without review.
cperciva is extremelyqualified to implement crypto, but not without review. I think it is wise that he has implemented a bug bounty procedure. He should make sure it applies to unreleased versions too so maybe someone will put an RSS feed of his SCM checkins into their RSS reader and try and catch bugs as he's making them. :)
It would be nice if he had the money to spring for paying someone else to look at all his changes, but alas... that stuff is expensive!
No professional is going to undertake a review on spec. The demand for software security is too high; most of us have our pick of interesting projects that will pay whether we find something or not. We're not unique in that respect; top iPhone developers won't work for you on spec either, not because spec is evil, but because the economics don't work.
Furthermore, you can pay $1000 for XSS bugs and random memory corruption flaws in browsers because fuzzers can find them, because they're luck-of-the-draw findings, and because people are hammering those things whether you pay them or not. But $1000 doesn't pay for a day of qualified review, and no qualified reviewer would suggest less than two weeks for something like Tarsnap.
However, since Colin presumably doesn't want to raise his prices to pay for actual review, it is encouraging that he is at least going with bug bounties. These, at the very least, gives us a good excuse to assign them as fun things to do for graduate students with some hope that one will want to procrastinate so hard that they will actually look at the code.
Also I think any reviewer who wanted to get paid would not start with Colin's code as an easy place to find bugs.
This is one reason why I'm going to be providing bounties for more than just security bugs. Even if people don't expect to find security bugs, they might find a typo in a comment and earn themselves a $1 tarsnap account credit -- and as demonstrated here, simply looking at source code can result in finding bugs even if you weren't originally looking for them.
Clever. Especially since "normal" bugs occasionally have that nasty habit of turning into security relevant bugs.
It will be interesting to see how close you can manage to get something resembling good review on a budget. Hopefully other people who are in similar low margin code businesses will keep an eye on your experiment to see how it works out.
Thanks for being so open about how you're trying to make things work. I hope you'll be publishing all the awarded bounties? (I suppose I should just wait for your follow-up entry.)
I think there's a limited market that would pay for this now, you say you would, other users say they would. But this is not something he can offer to only some users as an extra feature, so all his users would have to be comfortable with it. From a business perspective, I imagine he has done testing to figure out what price is right and from a security perspective it would probably is not too unreasonable if he just waited until he had enough volume to allow this to happen with only very limited price increases or at his current margins.
In the meantime, the bug bounty + very qualified developer strategy seems like a reasonably sensible option while the service is presumably, still in its growth phase. I guess we'll find out.
http://news.ycombinator.com/item?id=1183757
(not that I know anything about crypto)