Babbage didn't want to put the program and data in the same storage device. The program was a chain of cards; the data store was rows of number wheels mounted on a big cylinder for addressing purposes. Probably the closest production machine to Babbage's machine was the IBM Card Programmed Calculator, which we would not consider a computer today.
All thinking about this issue has to face the fact that there were no good memory elements before magnetic core. Delay lines were slow and sequential, drums were slower and sequential, and Williams tubes were faster, random access, but expensive per bit. Core memory wasn't cheap; it was about a million dollars a megabyte in 1970. "Just put the program in main memory" wasn't a good option until halfway decent memory hardware was developed.
All thinking about this issue has to face the fact that there were no good memory elements before magnetic core. Delay lines were slow and sequential, drums were slower and sequential, and Williams tubes were faster, random access, but expensive per bit. Core memory wasn't cheap; it was about a million dollars a megabyte in 1970. "Just put the program in main memory" wasn't a good option until halfway decent memory hardware was developed.