Configuring applications to run with elevated privileges on-the-fly