This is correct. When you consider that the intent of a historical body is hard to divine and open to all kinds of bias, it's not so obvious that it's the right way to interpret the document.
A key problem for the brand of originalism is that it is mainly used by people who see only their Neo-liberal bias in the history of the constitution. The "free market" conservatives are particularly guilty of this, as the founding fathers were not free market capitalists like ayn rand.
If only we had some notes about the debate and discussion that occurred during the drafting and ratification of the Constitution by the person that wrote it.
* Or maybe books containing short dissertations about the various aspects of the said document, again by the people that were deeply involved in the drafting and ratifying the same document while attempting to persuade the states to either vote for or against the document.
# Even better would be some of their correspondence where they discussed the document.
& Then maybe, just maybe, we might have some idea what their intent was.
*- Madison's Notes.
#- The Federalist and Anti-Federalist Papers
&- I'm pretty sure we have lots of the correspondence of many of the Founding Fathers.