I toyed with implementing Swift on the JVM (which is very doable by the way). The problem I've found with Swift is the minute size of the standard library. This has forced a large amount of Swift libraries to make calls to Objective-C/Cocoa classes libraries which makes it very non-portable. I haven't checked any time recently, but if a large, stable, swift-specific stdlib were to emerge then I believe it could definitely rival those other languages. Until then, most libraries will probably be Apple/OSX specific (not to mention Windows concerns).
I was going to make it use GC and make ARC-specific constructs a noop (I hadn't decided whether I wanted "weak" to be a JVM WeakReference or not, I did not get that far).