Shadow environment secretary Rachael Maskell called on the Government to mitigate the effect on the food industry of the UK's decision to leave the EU.
But Mr Gove hit back, saying the sector had "flourished" in recent months before inviting Environment Secretary Andrea Leadsom for a "knees-up" in a brewery.
Ms Maskell said: "From Marmite-gate to the Toblerone gap, we have had the rising prices across the food industry as customers are paying more for food while those working in farming and food production have been hit even harder and it's getting worse.
"What is the Secretary of State doing to mitigate against this?"
Mrs Leadsom, replying during environment questions in the Commons, insisted the food and farming industry was "incredibly thriving".
"Our food innovation is second to none," she said.
"We produce more new products every year than France and Germany combined.
"Food inflation is low. It continues to be low and we are seeing a very thriving sector improving on exports, up this year, and we are doing everything we can to create a sustainable environment for the future."
Ms Maskell then claimed that food "is inflating at 5%".
"This is on your watch, your responsibility, your crisis and people are struggling now," she said.
Mrs Leadsom replied: "Well, that was rather nonsense if I may say so. In fact, food inflation, food prices have been dropping. They peaked in 2008.
"Food prices do move up and down."
Mr Gove, who campaigned for the UK to leave the EU, later criticised Ms Maskell's position on the issue.
Addressing Mrs Leadsom, he said: "Food and drink production has flourished under your leadership.
"As we have just heard, record levels of hard cheese and sour grapes are emanating from that side of the chamber.
"In my own constituency, the Hogs Back Brewery, a very successful micro-brewery, is doing a roaring trade.
"Could I invite you to join me for a knees-up in my brewery, something the other side of the House could never organise?"
Mrs Leadsom replied: "Yes - absolutely be delighted to do that."