Indeed this is the stuff, or some similar sort of bonding sealant. Clean the bumper first with some lighter fluid and possibly key it up a little.
Polyurethane sealant is probably your best bet tho. All the TTE Gen3 MR2 front, rear and side skirts are stuck on with this stuff and im sure they wont ever come off..