The answer to your question is yes. Since each "string" (resistor plus however many LEDs you put on that branch in series) needs the same amount of current, and the input voltage is fixed, each string draws the same amount of power (since P = VI). This means you want to minimize the number of "strings" by putting as many in series as you can, then putting the "strings" in parallel.
For 4 LEDs at 3.2V each, go with 2 parallel strings of 2 in series. If you need 6, go with 2 strings of 3. Note that you'll need a different resistor for those two configurations. Battery life will be essentially the same. In the latter scenario, the resistor burns less power. In the former, the resistor takes up the power that would be taken by the 3rd LED.
As you've noted, you can't quite put all 4 in series. It'll work, but they'll be fairly dim. I also advise against connecting LEDs up with no resistor at all, as any fault will pretty much instantly destroy them all. The resistor, even if it's small, can provide a surprising amount of buffer if something goes wrong.