Hi teachme,
Like you said, the Galileo can work without using the SD card. The Galileo just needs a Linux Image to boot. Internally, it has a "small" Linux Image which allow the board to boot. You can use the board with that internal Image without any problems, but it has some limitations, for example you won't be able to perform WiFi applications since the drivers are not included in the internal Linux Image. Here is where the SD card take place.
You can download the complete Linux Image and store it in the SD card. The Galileo will boot from the SD card and run the complete Linux Image instead of the internal Image. You will be able to perform more applications, store files permanently, install packages, etc. That is the main difference. You can download the complete Linux Image (SD-Card Linux Image) from here: Galileo Software Downloads
Regarding the firmware, it doesn't refer to the Linux Image. The firmware of the board is the interface between the hardware and the software. It is recommended to keep the firmware version always updated because there are some features that are added and some bugs that are fixed with newer versions. The firmware is stored in the FLASH memory of the board.
I hope you find this information helpful.
Regards,
Diego.