Minutes of the Bank's Financial Policy Committee (FPC) meeting last month showed officials' concerns about the sector that accounted for 15% of outstanding mortgages and 20% of new home loan lending in the first quarter of this year.
They noted that the number of buy-to-let products where loans were worth more than 80% of the value of properties had risen by half in 18 months.
The FPC set out new policies on home loans a year ago to insure against the risk of a steep rise in indebted households and has been given powers to limit residential mortgage lending at high loan-to-value or high debt-to-income ratios.
But these applied only to owner-occupier loans and the FPC's latest minutes said that given this fact "it was possible for risks to financial stability to be transferred to the buy-to-let segment".
The Treasury has said it would consult on extending the Bank's powers on limiting higher risk lending to this sector.
The FPC said that it had asked Bank of England staff to collect further data on the market and "identify and quantify the channels through which buy-to-let lending might threaten financial stability" ahead of a September meeting.
It also asked for work to "determine what regulatory tools might be used... to mitigate any financial stability risks that were identified".
This would need to identify potential losses on buy-to-let loans as well as the wider impact of increased levels of indebtedness on the wider economy's ability to withstand shocks.
The FPC noted that the total UK household debt to income ratio stood at 136% and that while this had fallen gradually, it remained high "compared to historical and international norms".
It said it "continued to monitor closely conditions in UK property markets" amid house prices and activity in the market increasing again recently, and rates on many mortgage products being historically low.
The FPC, which published its twice-yearly Financial Stability Report last week, identified the housing market as one of the main risks facing the UK financial system.
Read more on AOL Money:
Where should you invest in buy-to-let?
Use your pension to be a pensioner not a landlord
Horror tenants destroy home and sell landlady's garden