Preheat the oven to 150C/300F/Gas 2.
Place the milk, cream and vanilla pod and seeds into a saucepan over a low heat and heat until steaming, but not boiling. Remove from the heat and leave to infuse for ten minutes, then remove the vanilla pod.
Grease a shallow ovenproof dish. Arrange the brioche slices into the dish.
Place slices of banana between the slices of brioche.
Whisk the eggs, egg yolks and caster sugar together in a bowl, then gradually pour over the infused milk mixture, stirring constantly.
Pour the custard mixture over the brioche and bananas and transfer to the oven to bake for 20-30 minutes, or until the custard is set and the brioche is golden-brown.
For the caramel, place the caster sugar into a clean saucepan and heat without stirring until the sugar has melted. Continue to heat until the liquid becomes golden-brown, then pour over the bread and butter pudding and serve.