However, things now aren't as good as they were then. The Empire fell, the Dwarves are angry and the Elves are sad, while evil Lord Doombad raises an army to conquer everything.
You, a group of heroes aspiring or established, must journey to the many parts of McGuffin the Great's empire, collecting the Relics of McGuffin, and bring them together to the Temple of Unpr'onounc'eable, where they can be used to stop Lord Doombad."
... the plot of pretty much everything ever. Honourable mention to Tolkien whose plot was 'get rid of the Relic of Lord Doombad', but that sort of trick can't be repeated too often.
Don't get me wrong, I don't think it's a bad plot. It certainly gives the protagonists the opportunity to travel around, learn a bit about themselves and the world, meet interesting new people, have some adventures... and that I'm fairly certain is pretty much what people are seeking to do with their characters when they play D&D. It should be possible to play D&D in a way that relates in some way to this kind of 'storyline'. Hopefully, without too much of a problem with regards to railroading, which isn't something I'm a fan of.
I think the trick is in getting the players to agree to the overarching conceit but allow them freedom within that. I've talked a bit about this before - particularly in this post about Questing in Elfgames. Basically the players need to agree that the PCs will take the existence of the quest - whatever it is, it doesn't have to be the 'Quest for the Relics of McGuffin' described above. It doesn't matter if they go and do a side-quest I suppose, but they need to keep coming back to the main plot. What they can't really do is ignore the quest. For this sort of idea to work, there needs to be some reason for the PCs - any PCs - to be doing it in the first place.
This is the most difficult part of the whole process and it is the point where character backgrounds, quests, mentors and all these questions about linking the PCs to the world background fit together. In fantasy literature it is often some secret that the protagonist(s) learn that convinces them to embark on some quest - Frodo finds out the nature of Bilbo's Ring, Garion learns of his heritage, Eragon learns something of the world and its mysteries from Brom, and so on. Other times it's more to do with personal relationships, loyalty, or just straight-up cultural conditioning. Pug goes on his adventure mainly because of loyalty to Duke Borric, and in 'portal novels' such as the Fionavar Tapestry or the Chronicles of Thomas Covenant, the protagonists generally fall in with whoever they meet first - which of course is how Edmund ends up on the 'wrong' side in Narnia. The stories continue because there some drive on the part of the characters that keeps them fixed on the plot.
There's nothing to force PCs to stick to the quest formula, except the players' desire. This kind of game must depend on the players' willingness to engage with the conceit - but then so does hexcrawling or megadungeon-bashing or any other kind of 'formula' for how play is meant to progress. In a dungeon-bash, the PCs pretty much agree that the procedure is "go and bash the dungeon the DM has created"; in a hex-crawl, the PCs agree to wander about sampling the environment that the DM throws at them; likewise in a 'quest' the PCs have to agree to the conceit of the game, that the quest is important. However, it's maybe not so strictly 'procedural' as some other forms of the game, it definitely has an element of 'story' involved and the PCs would need to agree to be guided by the DM to some extent here. That's perhaps where it gets tricky - there's no way in the rules of B/X to generate big plots, quests, mentors or character backgrounds, and it's not easy to see where such things would come from - unless from Jens's Narrative Generator and similar devices. The best that the B/X rules can offer is the notes at the back of Basic for dungeon-creation. On p.B51, in Chapter 8: Dungeon Master Information (the chapter on designing scenarios), Section A is called 'Choose a Scenario', and gives a list of suggestions.
Scenarios:This is something like a quest-generator... sort of. It at least provides the plot background, but goes no way to linking it to any notion of character background. So, while it goes a little way to providing something that I'm after, it certainly doesn't provide any kind of motivation. Some of the descriptions hint at possible motivations (for instance, the PCs might visit a lost shrine to remove a curse) but there's not enough to hang a system on.
1. Exploring the Unknown
2. Investigating a Chaotic Outpost
3. Recovering Ruins
4. Destroying an Ancient Evil
5. Visiting a Lost Shrine
6. Fulfilling a Quest
7. Escaping from Enemies
8. Rescuing Prisoners
9. Using a Magic Portal
10. Finding a Lost Race
It seems to me that computer 'RPGs' (I can't bring myself to really regard most of them as RPGs but I'm prepared to say games like Skyrim seem to be pretty much the real thing) have an edge here. Players buy in to the plot in order to play the game, whether that's Assassin's Creed, Zelda or whatever. There is an expectation that there will be a lot of player freedom but also some more directed elements to move the story along. That's pretty much a given of the format. Maybe that's a useful line of enquiry for how to make this more 'procedural'.
There is more to come on this, I'm sure...