How to conduct an IoT pen test

Penetration testing was much like taking a battering ram to the door of the fortress. Keep pounding away and maybe find a secret backdoor to enter through. But what happens if pieces of the network are outside of the fortress? With the flurry of Internet of Things devices, is it harder to conduct a pen test with that many devices and end points?