I used to teach the first CS course at both Harvard and University of Washington. I didn't have this issue at Harvard, but at UW there was definitely a bimodal distribution in abilities. Roughly half the students had what I feel is a more visual / associative way of thinking vs. the focused structure you need to program.
The concept that the "non programmers" could never understand was the abstraction of functions. Like:
foo(a) {
// stuff
}
foo(b);
would confuse them because they'd get hung up on foo's parameter declared as 'a', but called with 'b'.
I took a few CS classes at the University of Washington in the 08–09 academic year. The intro CS classes had recently been redesigned [1] to be more effective. Anecdotally: I talked four of my friends into taking one or both of the intro classes (split evenly between science/non-science majors), and they all did well and really enjoyed them. I think how programming is taught at colleges is definitely a fixable problem.
The concept that the "non programmers" could never understand was the abstraction of functions. Like:
foo(a) { // stuff }
foo(b);
would confuse them because they'd get hung up on foo's parameter declared as 'a', but called with 'b'.