There is a pref that controls the minimum width now (browser.tabs.tabMinWidth). The default changed from 100px to 76px in Firefox 57. You should be able to tweak this to change the minimum width.