you are not allergic. borage belongs to the cucumber family and the stems and leaves have these tiny prickles. just wear long gloves. they reseed like crazy..they are edible flowers and bees love them.
further, the leaves can be cooked as pasta filling. mild taste of cucumber. once you have one borage plant, you will never get rid of them. i have both the blue and white variety.