Powerful Ad Hoc Data Processing Tools

"I want PowerfulAdHocDataProcessingTools."

As a developer, and a power user, I sometimes have a bunch of data that I want to process, and I don't want to have to run a proposal through all the layers of management, and perform a cost-benefit analysis, and form and run a project team over the course of months, just to get some data processed. What are the best tools available, for me to just crunch some data?

Ad-Hoc "Data Processing" tasks we might want to do:


Comments: See the new list above.

From what I've seen, SAS is a really strong tool for grabbing and crunching data in creative ways. ...if if you're not doing any math on it. (And, of course, SAS really shines on the math stuff.) But it's a pricy tool. -- JeffGrigg

In Unix, shell scripts and piping are often used to process data. Command line tools, scripting languages and custom programs can manipulate data. Script languages have progressed from awk, through perl, to python, ruby and others.

But the powerful Unix command line tools and piping are not commonly available on Windows platforms, and many GUI tools are not compatible with command line usage.

PowerShell is an extremely promising tool for the Windows DotNet environment.

(But there doesn't seem to be any equivilent in the Java environment.)

See Also: ExBase

Oh please. ExBase is not, by any stretch of the imagination, a "powerful" ad-hoc data processing tool. Comparing ExBase to (say) SAS is like comparing a steel-wheeled rollerskate to a Mac truck. Even for simple database-oriented tasks it is utterly eclipsed by MicrosoftAccess, which is still firmly in the in-line rollerskate territory.

More generally, importing the data into a RelationalDatabase would enable you to use SQL to process the data. This approach could be relatively powerful -- as "powerful" as SQL data manipulation (DML).

See Also: ExBase, DataManipulation, NimbleDatabase, DesktopDatabase, ModernizingExBase


View edit of May 4, 2009 or FindPage with title or text search