You mean software not running on a server? Well, if you've got a Java enterprise programmer's love for design patterns, you could do A/B tests using a Strategy pattern. And if you wanted to decouple that from your code, you could have the Strategies created by a Factory. And if you configured your StrategyFactory in XML then you would never have any A/B testing code in your code at all... and this topic is giving me flashbacks so I'm going to stop now.
The nuts and bolts of doing this in downloadable software are not extraordinarily difficult. Pick a unique random identifier at install time, report random identifier with reports of conversion to the central server. (Passing it as a query parameter when folks open your website from within the app is so easy it is almost cheating. You can also ask for folks for a "hardware ID" to generate their license key, or something similar.)
See the presentation Paras linked to if you need implementation advice.
The nuts and bolts of doing this in downloadable software are not extraordinarily difficult. Pick a unique random identifier at install time, report random identifier with reports of conversion to the central server. (Passing it as a query parameter when folks open your website from within the app is so easy it is almost cheating. You can also ask for folks for a "hardware ID" to generate their license key, or something similar.)
See the presentation Paras linked to if you need implementation advice.