In 1947, J Lyons and Co, a British company famous for its tea-shops, began work on the Leo computers. In 1951, the LEO I computer was operational and ran the world's first regular routine office computer job. Over the following years, LEO II computers were sold to about 10 other organizations, ranging from Ford to the British Government.
Amongst the early less commercial applications developed for it: "Leo II/5 had a good 'Juke Box' program and was of course famous for playing the Sailor's Hornpipe for The Duke of Edinburgh when he visited." - is there an earlier record of a music program?
The developers of the Leo computers published a wide variety of scientific articles on computing theory, including a paper which could be said to describe the basis of parallel distributed processing (neural networks).
Wikipedia has an excellent article on the subject: http://en.wikipedia.org/wiki/LEO_computer
itself lists the first Univac as being delivered to the United States Census Bureau on March 31, 1951. The LEO says something somewhat vague about the first commercial business app "rolling out" in November 1951. Does government not count as a "commercial application"? It's a close call at any rate; seems like it's all in what you define as "first". First working prototype? First delivery? First running a real world app?
It is difficult to obtain info on the "first computer" (commercial or otherwise). Partly it is because determining such is not as simple as one might initially think. The different honors include the combinations
of these two lists:
- First TC design (TC=Turing-Complete)
- First TC machine actually built
- First machine to actually use TC 
- First mechanical computer
- First electro-mechanical computer
- First electronic computer
- First transistor computer (in practice, there's probably been a gradual mix)
- First IC-based computer
Thus, there are at least 15 (3 x 5) mile-stones. The first TC design was Charles Babbage's mechanical machine around 1880-ish, IIRC. However, he never finished building it in his lifetime.
 The German machine was determined after-the-fact to be TC, although not actually used for its TC abilities at the time.
 To complicate things, we could also include designs versus actual implementation separately. There's also OS mile-stones, such as first multi-tasking machine.