Does it matter what item you get pre-quest or is all pre-quest items this happens with?
It seems to a bit all over the place.
Main quest line items, for the most part, are protected because the corresponding dungeons are locked..
Some quest items actually trigger the quest when picked up...
However most quest items for the misc quests can result in having this bug.. apparently the radiant quest system seems to be the cause... It places these items randomly, but does so before the actual quest is triggered.. Result: Quest cannot be finished, item is locked in your inventory as a quest item. The same issue can be seen on some "clear dungeon x"missions.. (quests can't be finished because the dungeon was already cleared)
Two asides: Though incredibly irritating.. none of the items will encumber you. And we might see a fix in in patch 1.3. (a note is made about fixing the radiant story system) So on a new game, hopefully, these issues (for misc quests) will be a thing of the past. We just have to wait and see.. Otherwise these problems will be fixed sometime in January with, as Bethesda stated, an incremental Quest fix.