Briefly, Information Theory is getting real world phenomena to behave like symbols, while Cybernetics is getting symbols to behave like real world phenomena.
If you want to solve a math problem, then a computer plus a proper algorithm will suffice. If you want to design a system that interacts with its environment and can achieve goals while maintaining homoeostasis, the name for that is Cybernetics.
"Introduction to Cybernetics" by Ashby has been made freely available in PDF form by his estate. A great and noble service for which I commend them.
http://pespmc1.vub.ac.be/ASHBBOOK.html