Where the word "edda" comes from is a bit of a mystery, but the most popular theory at present is that the word meant "creed". It seems to have been applied to Snorri's work first, and then later to the collection we now think of as the Poetic or Elder Edda.
The Elder Edda is the first and perhaps most important source for Heathens. There are others: history and archaeology, folklore, the sagas. But this contains the bulk of the remaining Lore of our faith.
The Eddur are not infallible. They were, after all, written by Christians, some time after the conversion of Iceland. That authorship, and that passage of time, must have affected what was written. Additionally, they're not a complete record of the myths of the "Elder Troth"; others are mentioned, tantalising, but are lost to us. (For example, when did Loki and Odin make the oath that bound them together? Why did Loki spend time as a woman? etc, etc.) Some lays, such as Lokasenna, may well have been written post-conversion by a clever skald, as an amusing piece about the naughtiness of the gods. Snorri's work in particular is loaded with problems, such as his literary framing device and suggestion that the Aesir were a people from Asia. But his work remains important, and we are lucky to have it.
As such, though the Eddur are the closest we have to a "bible" or holy text, we continually question them, dig through it, seek out what our ancestors believed. They are a spring-board to the gods and to our faith, rather than a set of orthodox beliefs one should hold.
The Havamal - saying of the High One, or Odin - is one of the most referenced lays. The first two sections contain advice for the Heathen person, and it is the closest we have to any sort of rule, law or guide of behaviour. Heathenry contains no commandments, just guidelines of proper behaviour. One is expected to be honourable, keep oaths, not murder people and so on, as in secular society. On top of that is the advice given by the Havamal, from which some people distil a set of virtues to strive towards. The Nine Noble Virtues are a popular example, but many Heathens find that lacking, not to mention focusing on some very odd virtues and ignoring more important ones. Do we really need a list of virtues, when we have the Havamal? No. This lay tells us what our ancestors valued, the advice that inspired them, the codes by which they lived. (Or aimed to live. No one pretends the Norsemen were moderate about their drinking all the time!)
It means a lot more to one when one believes in and actively worships these gods. Yet the Havamal in particular contains verses that appeal even to atheists, as they are focused on living well, rather than on promises of an afterlife.
Here are some verses.
Cattle die and kinsmen die,Havamal, 75 (Bray)
thyself too soon must die,
but one thing never, I ween, will die, --
fair fame of one who has earned.
"Much have I fared, | much have I found,
Much have I got from the gods:
What spake Othin himself | in the ears of his son,
Ere in the bale-fire he burned?"
"No man can tell | what in olden timeVafthruthnismal, 54 & 55 (Bellows)
Thou spak'st in the ears of thy son;
With fated mouth | the fall of the gods
And mine olden tales have I told;
With Othin in knowledge | now have I striven,
And ever the wiser thou art."
O'er Mithgarth Hugin | and Munin bothGrimnismal, 20 (Bellows)
Each day set forth to fly;
For Hugin I fear | lest he come not home,
But for Munin my care is more.