While the UK already has the lowest burden of regulation in the G7 nations, according to the World Economic Forum, businesses across the country have been demanding more.
Speaking at the BCC annual conference, Javid said: "This government is delivering on its commitment to free firms from £10 billion of heavy-handed over-regulation and build a more productive Britain. Hundreds of businesses responded to our Cutting Red Tape reviews and we are taking decisive action based on their experiences."
He added: "Through the Enterprise Bill, we are extending the scope of our deregulation target to cover the actions of regulators, going further than ever before to tackle troublesome red tape."
Government departments have been challenged to save business £3 for every £1 of the cost of new regulation.
Read more: Take red tape burden away from UK startup
The Federation of Small Businesses welcome the "one-in, three-out" rule, but warned the review should include the tax regime.
"This will help as small businesses are increasingly under pressure from a series of significant new challenges like the introduction of the National Living Wage and pensions auto-enrolment, all of them driving up the cost of doing business," said Sandra Dexter, FSB national vice chairman.
“However, there is still plenty of room for improvement. Today’s review excludes the complex and burdensome tax system, the most often cited example of onerous red tape highlighted by our members," she added. "Overall, deregulation should be proportionate, based on robust evidence, and with a clear small business focus to ensure it does not just benefit larger firms.”
And the CBI responded well to the announcements, acting director of competitive markets Tom Thackray having said: “Businesses welcome moves to drive down unnecessary regulation and tackle it sector-by-sector."
However, he added a warning that "the government must ensure that as many policies as possible are covered by the £10bn target, as significant exemptions risk undermining the credibility and impact of the wider deregulatory agenda”.
Specific actions include Defra publishing new business-focused guidance into the definition of waste and revised guidance for the environmental permitting regime.
Javid also pointed to the department of health and communities and local government leading a programme of work to make sure that multiple interventions made by public bodies that deal with care homes are targeted, proportionate and co-ordinated.
The department for climate and energy change will consult on whether to update regulation on energy storage, which could benefit consumers, businesses and the environment.
And to top it off, a new Cutting Red Tape review was launched today, which will look into unnecessary burdens placed on businesses by local authorities.