This might be slightly off-topic, but are DataFrames intended to be similar to R's data.frame, where the contents of the entire table are loaded into memory and operated on all at once, or more like SQL with the possibility of aggregate functions calculated incrementally? Since a lot of statistics can be calculated incrementally (OLS, obviously, but even something like MLE based on nonlinear optimization, if you allow for multiple passes through the dataset) R's approach really bugs me... even though providing the right tools for an aggregate function approach would be (I imagine) quite a bit more difficult.
DataFrames are in-memory. It sounds like you're describing what we're calling DataStreams, which are still a work in progress, but do already exist. And we're also building SGD descent for doing things like OLS incrementally.
