Rav Yosef said that this minhag began 600 years ago in Ashkenaz, Germany, by the Maharil. From there it spread to other communities throughout the Jewish spectrum, including Hassidic and Sephardic communities. The Vilna Gaon famously opposed this custom as he considered it too similar to the custom of the goyim on their holiday bringing trees into their homes, presumably Christmas trees.

This year, with the complication of Shmitta, Rav Yosef says one would not be allowed to decorate the shul with actual fruit of shmitta, fruit that has kedushat shviis, but cutting branches off of trees to use in the shul as decoration is allowed, based on the psak of Rav Ovadia Yosef (who paskened this way based on the Chazon Ish).

So get those shuls all green. Shvuos is just a few days away!