In a mixing bowl, whisk together heavy cream, powdered sugar, and vanilla extract until soft peaks form.
In a separate bowl, beat the cream cheese until smooth and creamy.
Add pumpkin puree, granulated sugar, cinnamon, nutmeg, ginger, and allspice to the cream cheese and mix until well combined.
Fold half of the whipped cream into the pumpkin mixture gently using a spatula.
Place a layer of cake cubes at the bottom of the trifle dish.
Spread a layer of pumpkin mixture over the cake cubes.
Add a layer of whipped cream on top of the pumpkin mixture.
Repeat the layers until all ingredients are used, finishing with a layer of whipped cream.
Drizzle caramel sauce over the top and sprinkle with chopped pecans if desired.
Refrigerate the trifle for at least 2 hours before serving to allow flavors to meld.
Notes
For an even more flavorful trifle, consider making it a day ahead of time to allow the flavors to develop further. Always use cold cream for whipping to achieve the best texture, and ensure the cream cheese is softened to avoid lumps in the mixture.