I thought about putting in zero, but I'm thinking of this in the mathematical group Z sub n, the natural numbers. That ring does not include zero, and it's as if it doesn't exist. One, on the other hand, is the multiplicative identity, so it is the background color, a factor of all the other numbers.

Certainly the ring `Z/nZ` has 0, meaning an additive identity (the image under the natural homomorphism `Z \to Z/nZ` of the 'true' `0 \in Z`). Do you mean that the group `(Z/nZ)^\times` of units of `Z/nZ` doesn't contain 0?

(I think that it's also confusing to call `Z/nZ`, or `Z_n` or whatever you like, the group of natural numbers; rather, it is a quotient of the semigroup (or semiring) of natural numbers by a normal subsemigroup (or semiideal).)

Okay, you passed me. I knew I should have pulled out my very old Modern Algebra book. I was thinking in terms of modulo groups. You can show lots of number theory with the sweater, granted that the number on the end is the equivalent of zero. I want a group under multiplication. Is that making any sense?

If you want a group under multiplication, the easy options are: (1) Integers modulo a prime number, not counting 0 mod p. (2) Integers modulo anything (call it n), not counting ones that have a common factor with n. (3) Rational numbers.

In this case you've got 1..100, and 101 is prime. So the numbers on (the front of) the sweater, mod 101, form a group under multiplication. But of course in that group you lose all the prime-number structure shown on the sweater -- you can't really talk about, say, "multiples of 3 (mod 101)" because 101 isn't a multiple of 3.