The US space agency is pointing its super hi-tech Webb telescope towards the "dead stars" to solve the galactic puzzle.
These objects can be 70 times larger than Jupiter so they can’t be classed as a planet.
But they do not have enough mass at their core to burn nuclear fuel and radiate brightly like a star.
Though the existence of brown dwarfs was mooted in the 1960s, they were only confirmed in 1995.
And boffins are still not even sure how exactly they are formed.
When a cloud of gas and dust collapses on itself in space, a star is born as temperatures reach 3 million degrees Celsius and fusion kicks in.
But because brown dwarfs do not have enough mass, they are prevented from collapsing further and the fusion never starts.
A NASA spokesman said: “Brown dwarfs muddy a clear distinction between stars and planets, throwing established understanding of those bodies, and theories of their formation, into question.”
They can act like stars but many drift alone in space, not surrounded by planets.
Brown dwarfs do not shine brightly like stars but emit the dim afterglow of their birth so they are best seen in infrared light – which is perfect for the Webb telescope.
NASA plans to study the stellar nursery of NGC 1333 in the constellation of Perseus which has an “unusually high number of brown dwarfs” which are less heavy in comparison to others.
Another research team from Universite de Montreal plans to use Webb to study a specific brown dwarf, labelled SIMP0136, which is one of the closest to our sun.
Astronomer Aleks Scholz, of the University of St Andrews, said: “In more than a decade of searching, our team has found it is very difficult to locate brown dwarfs that are less than five Jupiter-masses – the mass where star and planet formation overlap.
“That is a job for the Webb telescope.
“It has been a long wait for Webb, but we are very excited to get an opportunity to break new ground and potentially discover an entirely new type of planets, unbound, roaming the Galaxy like stars."
There has been suggestion an object – dubbed OGLE-2016-BLG-1190Lb – which is 13 times the size of Jupiter and located 22,000 light years away could be a brown dwarf.