Yes. You are right, to a point. You do occasionally see some fiddling like this going on - if you check adverts for businesses for sale, you see a suspicious number for sale with annual sales just a sliver below the VAT threshold.
However, the advantge isn't quite as big as you might think all the time. The smaller seller who can't charge VAT, also can't reclaim VAT on their purchases.
Let's say I have a business making widgets. I buy €50 of parts to make a €166.67 widget.
If I am VAT registered, I pay the seller €60 (€50 for the parts, and €10 for VAT @ 20%). I sell the widget for €200 (€166.67 for the widget, and €33.33 of VAT). I pay €23.33 VAT to the tax man (€33.33 which I collected less the €10 I paid out).
if I am not VAT registered, I pay €60 (€50 for the parts and €10 for VAT). I sell the widget for €200 (no VAT charged). The tax man loses out of €23.33, which I keep as profit.
If I am selling direct to consumers, then a small business is able to undercut this way. Just how much they can undercut depends on how much VAT they need to pay. A box shifter, buying gismos in €100, and selling them for €120, only saves VAT on €20.
However, if I am selling B2B, then if I can't charge VAT, my customer can't deduct the VAT. So, in my first example, my VAT registered business only pays €50 for the widget parts (Although the invoice is €60, I get a €10 credit on my own VAT bill). Similarly, if a VAT registered business buys my widget, they get a VAT credit, so only actually pay €166.67.
If I am not VAT registered, a VAT registered buyer cannot get a VAT credit, so pays €200 for my widget.