Simply put they are ‘Tamriel’s mystical tomes of knowledge that told of its past, present, and future’. For a full definition, I’m just going to quote from the UESP :
[they] are scrolls of unknown origin and number which simultaneously archive both past and future events. The number of the Scrolls is unknown not because of their immense quantity, but because the number itself is unknowable, as the Scrolls “do not exist in countable form”. They are fragments of creation from outside time itself, and their use in divining prophecies is but a small part of their power. They simultaneously do not exist, yet always have existed.
In the series up until Morrowind, they were mainly used as a plot device (a source of prophecies) and did not physically appear in the games until Oblivion. In Oblivion the Elder Scrolls were stored in a chamber in the White Gold Tower (in Cyrodiil). The final quest of the Thieves Guild sends you to steal one of the Scrolls. After the heist, they attempt to do a stocktake, however the number and placement of the Elder Scrolls seems to fluctuate, which makes the task impossible. Finally in 175 (fourth era), the Elder Scrolls are scattered across Tamriel, it is not known how this occurred.
In Skyrim, three Elder Scrolls were discovered. The first Elder Scroll is found during the main questline (quest: Elder Knowledge). This scroll allows you to learn a new dragonshout which is needed to defeat Alduin. The other two Elder Scrolls were added for the Dawnguard DLC, the first is carried by a Vampire named Serana (met during the quest Awakening) and the second obtained from the Soul Cairn during the quest Beyond Death. Reading the scrolls points to the location of Auriel’s Bow, which is needed to defeat Lord Harkon (at the conclusion of Dawnguard’s main questline).
In the Elder Scrolls Online, they are housed in secure temples in Cyrodiil (PVP) and they give a bonus to the controlling faction.
(I nearly wished I hadn’t asked this question :P)