Not disagreeing, but observing Apple has a big scale problem: They sell millions, hundreds of millions of things. They are remarkably small in the clue department, so the clue density remains high (this is the MIT theory of clue: the more people you have the thinner the layer of clue has to spread until almost everyone is virtually clueless)
View it from their point of view: a spigot will deliver 20,000 important, believable problems to everyones desk like a mailman opening the letter-to-santa sack in December. Every desk: Which one(s) do you look at first? Note, all of them are from people of good intent, not fools. The fools are railing on bulletin boards.
BTW I have met Apple engineers who write things. Important things, like the TCP/IP stack for the iPhone. There are two of them. They can't write amazing code, and triage 20,000 bug/improvement inputs at the same time. Sometimes, they're at standards meeting handling fools like me.
Sure, there are ass-hats, but the vast bulk of the time Apple is drowning in good input, and has to triage. The example listed here probably affects 1,000 to 2,000 developers, and a critical bug probably has to affect 150,000 people to get traction.
or you can spend $99 and prioritize your problem.
I had this problem with Microsoft and X509 about 15-20 years ago, (selection of which client certificate to identify a user when there is a certificate subjectName collision) -There was precisely one expert able to fix it, I was ringfenced from them, talking to a consultant and I had the great joy of hearing how much it would cost him to even TRY to get their attention on this problem, which btw, affects the US Government, so its not like it doesn't have "scale" behind it.
This is a problem you get, if you have hundreds of millions of customers.
I often wonder how Apple can keep posting these quarterly results of a gazillion dollars and still not reply to my Feedbacks for months or years - of things that are very clearly broken!
As a dev, I‘m paying them ca 2k per year in hardware costs. Sure sounds like the incentives to really fix this would be there.
Apple often claim they are among the most innovative companies. They don’t seem particularly innovative in this space. Eg. the Feedback app looks like it’s been made by a single person over the span of a week.
> I often wonder how Apple can keep posting these quarterly results of a gazillion dollars and still not reply to my Feedbacks for months or years - of things that are very clearly broken!
I mean, you said it yourself:
> I‘m paying them ca 2k per year in hardware costs.
Despite the fact Apple hasn't responded in years, you still pay them 2k a year. Companies exist to make a profit, and Apple has seemingly correctly identified that they can skimp on this area and people will still send them money.
1. From your perspective you’ve filed an issue that perfectly describes the problem. 90% of the time the engineer doesn’t actually have enough information and needs follow up. 90% of that time the bug just isn’t actionable because it’s somehow reliant on the user’s configuration OR requires extra logging to be enabled that wasn’t.
2. There’s so many communication layers between you and where the ticket needs to get to to answer your question. That routing takes time. Apple has a big enough challenge managing this just from internal bug tickets.
3. Organizationally Apple is in a no win state. One engineer having a bad day and responding poorly on a ticket impacts their image and generates a news cycle. So now every response needs to be filtered through someone who’s only job is to manage public engineer comms. This is a low value spend for Apple.
4. Some responses might be something like “this will be fixed next release “. Culturally that kind of response would be an anathema at Apple because of the secrecy.
5. Point 4 interacts with point 1 - they don’t want engineers talking in random environments to customers (WWDC is a very controlled environment, the engineers get some training, and it happens just after they’ve announced most things for the year so the opportunity for accidental leaks is smaller).
6. Engineers have a fuckton of responsibilities already. You have to screen a bunch of bugs, develop new software on insane timelines, manage tech debt, keep up with internal trainings, etc.
Could they do a better job? Are there cultural things at Apple that inhibit these things? Yes. I’m sure Apple is trying to fix some of them. But some of these things are trade offs between how Apple chooses to run their business and there aren’t easy answers (if there were these problems wouldn’t exist).
On a personal note, as someone who’s managing a fraction of the volume some of their products get and trying to stay very involved with the community, it’s extremely distracting and demoralizing at times (some users can be very entitled to the point that they’re a distraction and taking away value from the community outreach that’s happening).
Is you comment directed at Apple or applies to every company doing support? Because I can have good support - and I had - from others, just never from big names, Apple or whatever. So sorry, I don't buy it. Good customer service can obviously be done, if the company cares about it, and Apple just doesn't care. Or Google. Or Facebook.
This is just a thought experiment, but if we replaced each of the "big names" with a collection of smaller companies would we be able to get the same level of innovation?
I think so. I believe it's being good at something that makes a company grow, not the other way around.
Just looking for an answer to all those comments asking if the problem is tractable.
I also don't see what's innovation to do with customer support. Are you trying to say that one can only be innovative when offering shitty customer support? Or that the innovation IS the shitty customer support? I'm confused.
I think you are mistaking B2C feedback with feedback from developers, who Apple relies on to make apps and provide exactly that kind of feedback.
Whenever I talk to an Apple employee on Twitter about a bug I’ve found, they tell me to file a Feedback. At WWDC, we are also often told to file any unexpected behavior and feature requests as Feedbacks.
No I’m not. Developers often offer only marginally better feedback which you can observe if you’ve ever triaged bug reports within Apple or elsewhere. You individually may do a fine job but in aggregate you have many of the same problems.
Apple sells around 300 million devices a year. So has somewhere north of a billion active devices at any given moment. ~1k new apps ship on the app store each day, so >365k per year.
Apple has ~154k employees.
It's a completely intractable problem. No single person or developer matters in the smallest way. Any complaint or bug that doesn't break at least 5 digit numbers is invisible unless you're paying for Apple's engineering time. It's a function of the numbers in play.
I don't think the problem is intractable, but I also don't think the problem is going to be fixed very soon. It's like google. I doubt any googlers here like it when people use "complain on HN" to fix "my account was locked" stuffups. The bottom line is that with 125,000 employees for 1.5 billion users, they don't have much choice unless people pay. (I paid for google one in part to have at least SOME paid access to helpdesk and so far, they've been very nice)
Apple is very focussed on profit. There often is very little profit in this space and the more time it sucks to decide #WONTFIX the more it cost you. In that model, which is not a good model, but in that model, doing less engagement pays back better if you make really really good product with low overall bug/problems.
For more normal people making normal things, its suicidally bad. Ignore your customers at your peril.
The responses in the other threads about paying for additional support tickets are I think the solution you are pointing at.
You'll be paying 2k per year in hardware wether Apple invests more in support or not so I'm not sure it works as an incentive. Your option would be to drop out of iOS development and move to windows or linux, but that's probably not a statistically relevant portion of devs from Apple's perspective.
Then qualified engineering time (having people able to write that reply about the extra parameter) really doesn't scale, so price-gatekeeping the service also seems logical from a business perspective.
But those are bugs in their APIs, often very blatant ones. Would you feel good about paying for that to get fixed? IMO Apple needs to take care of that walled garden they are creating.
I see it as a fact of life: there will be bugs, and wether your particular bug gets prioritized will depend on many factors otherwise out of your control.
Apple will have a never ending list of legitimate bugs and improvements to work on, so it comes down to wether you really care about the issue you’re facing or you’re fine to leave it to the odds to get solved or not.
I got the impression from the post that the problem in this case, like in many other cases with Apple, was a lack of documentation. Why would this not be documented? It would save everyone time, sweat and tears if the documentation was OK.
I'd like to believe that's the issue but my own experience leads me to believe they really don't care about external feedback. Apple heritage is "the customer doesn't know what they need" which is at odds with customer feedback. It's almost as if they view incorporating feedback as weakness which is fair enough in the earlier years.
The short answer is only vestigially. TCP/IP has diverged into multiple competing versions across the OS world. Things like Multi-path TCP, BBR and other congestion control logic, the incorporation of things like Happy Eyeballs (v6/v4 preferencing) and some basic modelling of end-to-end RTT/jitter in the kernel is all pretty much post-FreeBSD. "write" is the active word here: Leffler and Joy "wrote" the TCP/IP stack for BSD. Lots of people are "writing" TCP/IP code. Maintaining? Modifying? Improving? Augmenting? Bugfixing?
Feedback Assistant is not a support channel. It also is not a new name for Radar. You can find support channels under “Requesting Support”: https://developer.apple.com/bug-reporting/
You are not limited to 2 technical support incidents per year. You can purchase more. I think it’s pretty fair.
I’ve only ever filed one or two of these “technical support incidents” but even though it sucks to pay for support — they have some very good engineers answering these tickets, and in one case they got a bug fix for an API shipped pretty quickly.
> Additional TSIs are available for purchase in either a 2-pack for 99 USD or 5-Pack for 249 USD (or in local currency where available) in the Code-Level Support section of your account. TSIs purchased separately expire one year from the date of activation.
Apple: Additional TSIs are available for purchase in either a 2-pack for 99 USD or 5-Pack for 249 USD (or in local currency where available) in the Code-Level Support section of your account. TSIs purchased separately expire one year from the date of activation.
Seems like you save a few pennies per incident with two packs unless you really need an odd number.
Forcing multiple-unit pricing likely increases sales. You probably came to the purchase page just needing one. But you’re probably 𝖽̶𝖾̶𝗌̶𝗉̶𝖾̶𝗋̶𝖺̶𝗍̶𝖾̶ motivated enough to pay extra for a bundle of two if you have to! You’ll justify it by telling yourself “I might need the second one later”, but I’ll bet a lot of them end up going unused.
Raising the entry price slightly perhaps also makes people think a little harder about whether they really need to use a TSI or whether other support channels (Stack Overflow? Reddit?) might be more appropriate.
I could imagine a corporate customer who might be slightly less cost sensitive but have employees that would prefer not to go through a purchase order approval process multiple times might be where purchases like that get made
For what it's worth, bugs that Apple engineers file inside of the company often go into the proverbial "Bit Bucket." When I worked at Apple many moons ago I experienced a really horrible bug when performing a system update. I meticulously recorded all of the context and submitted it directly to the owning team. That team proceeded to ignore my bug report for over a year and then kick it back to me with, "It's been a while. Maybe it's fixed. Try going through your repro steps again with the same hardware and let us know what happens."
By then I had swapped out said hardware with more recent stuff I was doing development on.
How many people were impacted by that bug in the meantime? No clue, but I guess it didn't hit Apple's bottom line hard enough to warrant anyone on the owning team even bothering to follow my repro steps themselves.
For all the hate Microsoft gets, this is one area where they've got it right.
Microsoft's dev support framework, while not directly interacting with MS engineers (usually), the community of MVPs, Regional Directors, and even now the occasional Dev note in the opensource work MS is doing on github, has lead to a great ecosystem of support. This trickles down to Stackoverflow, the old Alt.Net crowd (and the associated .NET user groups that still persist), and the MS dev blog'o'sphere mean this type of issue is much rarer.
*edit: oh and I forgot the UserVoice pages, where the community can vote on issues, and they'll get assessed by the product teams.
I had the exact opposite experience with UserVoice pages, at least the one for Accessibility. Someone requested that MS integrate my startup's reading technology into MS Word/Outlook/etc on the Accessibility UserVoice. This suggestion gained steam and was the top-rated suggestion for years. MS never engaged with the suggestion, and when I later asked an accessibility person why this was, they told me that the UserVoice is staffed by outside contractors, and that it's not one of the more important places they look for feedback. This was consistent with what I saw, which was that they didn't engage with the most popular suggestions on the Accessibility UserVoice — they just ignored them.
I think it's pretty lousy that they set up a page to solicit feedback (especially accessibility feedback) and don't pay attention to it. It causes people to waste time describing issues, asking for help, and voting on other suggestions. A year or two ago they got rid of the UserVoice page, and they now tell people to use Twitter to make suggestions.
I posted an obscure MS C++ compiler bug to their Web feedback thing, and it got acknowledged and fixed relatively quickly. Whining and complaining over on the r/cpp subreddit also tends to get the polite attention of the relevant Microsoft people (some of whom are mods).
They eventually got it right. I remember being a sysadmin circa 1999 and having to pay to file a bug report with Microsoft. IIRC if they determined it was a real bug you got your money back.
Independent of Apple, I think we need an industry wide of saying "I'm not an idiot, this bug report is real". I've been on both sides (in a moderately used OSS project). The main problem is that the attending doesn't have a good way to filtering the noise from the signal. As a result, the likes of Apple (and the other FAANGs) implement these aggregate-and-discard blackholes for bug reports. "Only a 0.1% increase in crashes? Ship it" is the way the story goes sadly.
>The main problem is that the attending doesn't have a good way to filtering the noise from the signal.
Even if you could filter the noise; there's still a non-zero chance it won't get prioritized. I imagine in Apple's case you could have very real bug reports that only affect 500,000 people and they can't allocate time to fix this. I think there's a tendency to anthropomorphize companies (and OSS projects) as an all knowing person who can fix every problem and has infinite time. In reality there might be 10 people (or in OSS, just 1 person) in the company uniquely capable of triaging your issue in under an hour. The rest would be just as lost as you would be in a new code base and they have to weigh fixing your bug report vs the other opportunities they have (and I'm sure at Apple, closing Radar tickets wont get you promoted).
Maybe after two years of engineering experience you get one “this is real” token to use on an issue. If you use it and the issue actually is real you get the token back. If not you have to do two more years of engineering work to get a new token.
As an OSS maintainer, the way to show this is to show up with a full reproduction. "Run exactly this and it crashes".
Can't reproduce it? Well... you can of course still report it but "I'm not an idiot" is a pretty weak signal. Everyone gets lost in the weeds, and debugging a problem only for it to be an issue in some unstated part of the project is very exhausting.
As someone that has recently opened a bug report on a popular open source project, with a very simple and 100% effective reproduction process, I had a back and forth with the maintainer telling me to "what happens if you do X instead?"
What is the point of me providing an example case if the ones that know the internals of their code don't try to reproduce my bug?
Yeah, I know OSS devs do not have time/are unpaid. My point is however complete your bug report is, nothing guarantees it won't go stale without anyone having taken a real look at it. In my experience 90% of bug reports, on OSS or commercial products, die this way.
It's definitely broken for one-off reports, but in the last beta period (for iOS/iPadOS 15), they actually made some changes. There are user visible "Resolution" and "Recent similar reports" fields. I actually submitted a few feedbacks and saw them change from "Recent similar reports: none" to "Recent similar reports: more than 10" and then "Resolution" actually did change to "fixed in an upcoming release of iOS". At least two feedbacks I submitted got responses to them, which actually got me an email asking for more information on one.
I don't think the system is perfect by any stretch. But it definitely has improved, and Apple does read them. The catch is that for esoteric issues -- especially ones that say "Recent similar reports: none", publicizing them and asking people to submit feedback if they can reproduce the issue will likely help prioritize it.
Esoteric crashes /are/ harder to diagnose. Minority issues are also lower on the totem pole for any org. I don't want to excuse Apple here -- it does suck, and it does need to keep improving, but last beta round was seriously an improvement over the past.
Only once I've got any response to Feedback. After getting it I never opened another case again.
Can't recall what exactly was the issue but after macOS update, some bundled software stopped being so friendly and required more actions to achieve a goal. Some built-in functionality stopped being more useful than 3rd party products.
I've opened a case to try to tell Apple in what way their update has broken my productivity, pointed out what was needed to get results before the update and how more labor user had to do after the update.
Got the most Apple answer possible: "this works as intended".
Just another example of Cook's hostility towards users to make Apple more attractive to investors.
Have you tried doing the same for Google? Or any other company at this scale. Even though I pay for Google One/Drive/Whatever they call it now, I routinely have data loss level issues and get the "that is not my problem" response.
This seems better than the experience at any other big tech though. Even filing a bug is considerably harder on a Google/Amazon product because there's no unified Feedback Assistant app. The experience of firing a bug report off into a black hole and never knowing if/when it was seen is universal as well.
It is frustrating but hardly newsworthy or a surprise.. if you feel all feedback requests should be personally investigated by devs/PMs and replied to, maybe talk to the support teams and get a sense of the volume of these reports? It really is impossible.
Having no way to file a bug is better than pretending to have one which just gets filed to /dev/null. At least that way I don't waste time writing bug reports.
1 that became irrelevant without ever being fixed (an iOS 7 specific problem).
5 fixed bugs in vim bindings in Xcode.
4 clang/libc++ bugs fixed upstream which eventually got pulled into the Xcode version.
4 other bug fixed
Reporting vim binding issues during the Xcode 13 beta was a good use of time, but outside of those I'm 4/21 for reporting something to feedback assistant/radar doing anything.
I'm not sure what Amazon does but when I worked at Google all effort was basically in battening down the hatches to make sure there was absolutely, positively no way to reach a human being anywhere in the company unless you were a business partner of some kind. I stopped answering the phone because every call was someone randomly typing extension numbers out of sheer desperation.
This is one of the main reasons I don’t develop for Apple’s platforms any more. I’d love to write Swift and leverage all the technology in their ecosystem but when you hit a wall it hits hard and you waste hours, days or weeks searching and hoping someone else hit the same issue and wrote about it.
Their docs are dire, their feedback system is terrible and they’ve made essentially zero improvements in the 7 years since I shipped my first app.
Web dev has its own frustrations but at least you can find an answer (or ten) quite quickly when you get stuck.
As a side note I had a play with the WeatherKit REST API for fun during WWDC this year and got totally stuck. Had a labs session with two engineers (who were great) and it turned out it was an issue with missing docs and seemingly none standard JWT signing process. Their docs (at the time) made no reference to any of this. So things really haven’t changed.
It's really a terrible experience. Especially since no one else can see it and simply upvote it. Instead everyone has to do the busy work all over again.
Last time I tried to use Feedback Assistant, it was basically impossible to report an issue with the Apple Music app on my Android phone. Scrolling was totally broken in both Chrome and Firefox, so I couldn't scroll through the list of things to file a bug against and find the right one.
(I don't know if that bug still exists because now for some reasons apple won't let me access Feedback Assistant unless I give them the CVC from my credit card and my phone number to "verify my account"
I can offer some thoughts as to why this is the case:
- Apple engineers are literally drowning in Radars.
- Your F.A. Radar will arrive so late to an engineer (after being filtered by QA) that the engineer is working on new features and marks F.A. bugs as low priority.
These are likely the exact issues. For the past several years for OS releases Apple sets priority deadlines. There's a date where all P1 bugs need to be closed/moved, and so on until near the scheduled release which is "zero bugs". All that means is bugs that didn't get fixed move into the next OS milestone.
These deadlines basically preclude late breaking or low priority bugs to ever make an engineer's schedule. The only way is if an exec pushes on the bug or if it turns out to have higher impact than originally thought.
Before Feedback Assistant bugs could come in from AppleCare support or developer relations. These could be some of the worst bugs to get assigned because they would be filled with extraneous details but not a crash or spin trace that would actually be useful.
Not only Apple, I send this email to EU Commissioner,
I heard numerous cases when FB support completely ignored cases. There is even a post on Reddit of a guy, who managed to resolve his case by buying an Oculus gadget. Just today I heard a case with Upwork. I am trying to understand here, what is this? A corporate policy? A resource shortage? When you buy a car and it is broken, you can go to court and seek resolution there, but when you have a social account with a large user base or an Upwork account that brings money, where do you go?
Shouldn’t companies have a standard flows to support people? Would you as EU impose some rules?
I’ve had the same experience reporting bugs every single time. You never ever get a response and simple bugs languish for literal years with no action. Eventually, they sometimes get fixed after years and you wonder if your report had any connection to it whatsoever. It definitely adds to the “Apple doesn’t give a damn about users or software quality” impression I have gotten more and more over the years
I think Apple pushes out whatever is on their internal roadmap, and if it works for most of the users, it's fine. If not a serious issue, it may never be fixed.
There are somewhat serious/annoying issues that have been there for multiple years but are mostly relevant for power users, or issues that do not contribute to the bottom line, and thus will never be fixed.
Even if their own apps do not work together nicely, tough luck.
Hope that your issues will be fixed in the next major OS release.
I think that it is a problem with their mode of operating and the fact that fixing the majority of bugs will take longer than pushing out a v1 of a new product or service.
Also, fixing these bugs will probably mean that teams have to talk to each other, and I really have no idea if that is a reality at Apple. Given the last few software releases it doesn't seem that way.
I have a bunch of issues with Monterey that, while some are VERY annoying, do not prevent me from being productive. I also have zero expectation they will be fixed.
Fun fact: 3 major releases of TvOS have functioned such that if you navigate to App Store/search and hit siri “Xyz app” Siri (global) appears and tries to take you to said app.
If you want to use dictation, you must swipe down and select the text input field. You know, because navigation to search wasn’t enough context.
My laptop somehow overwrote its primary disk encryption keys (i.e. It basically crypto-shredded all my data). I filed an issue with Apple and was met with radio silence for a year. The response that came back was effectively, “do you still have that device, and is it still broken?”
Does anyone have any ideas on how Apple could solve this, without just hiring thousands of people? What process or system could be used to triage and reply to tens of thousands of these types of tickets without a corresponding linear increase in staff?
Feedback and bug reports (exception: security) should be public and users should be able to vote. Apple then could prioritize much better.
Also, take a look at their user forums: They are full of bug reports with hundreds of users having the same problems - and nothing happens. I really wish they would react much better with official acknowledgments, references to open bugs, workarounds, etc. It is depressing how much collective time is wasted.
They make enough money to hire all those people. Maybe pull in some extra contractors until the queue is manageable and then staying on top of it wouldn’t be as bad?
I reported `/usr/bin/CLANG` odd behavior on case-sensitive data volume due to case-insensitive system volume on Mac OS X.
Then they replied with "Work as expected; No plans to make the system volume case-sensitive".
There is something worse than what Casey experienced.
I filed a feedback on the Contacts API that you can't add a piece of information to a contact card. Literally. You can only read the entire card, set stuff on it (not ALL the stuff, just some random subset!) and then write it. What happens to all the data you can't set and can't read?
REMOVED
This means their Contacts API, if used to add any information, leads to data loss.
The reply? "Works as designed". I explained to them that this is catastrophic data loss and that how it was designed isn't really relevant. Still got "works as designed".
He's right. I submitted a radar[0] in the Xcode toolchain with a very simple repro and it was never acted on - a compile would fail with an error when there was none. It killed our CI system.
I’ve rarely had my Radars receive any attention unless they were coupled with a TSI. Probably makes sense based on the volume of bug reports that Apple likely receives.
The ones that were accompanied with a TSI received attention in 1-3 weeks which seems pretty quick.
Apple does not need active feedback mechanism. It just needs a list of gotchas to avoid for their next generation of products. Their customers of previous products can go f off or buy the new products.
View it from their point of view: a spigot will deliver 20,000 important, believable problems to everyones desk like a mailman opening the letter-to-santa sack in December. Every desk: Which one(s) do you look at first? Note, all of them are from people of good intent, not fools. The fools are railing on bulletin boards.
BTW I have met Apple engineers who write things. Important things, like the TCP/IP stack for the iPhone. There are two of them. They can't write amazing code, and triage 20,000 bug/improvement inputs at the same time. Sometimes, they're at standards meeting handling fools like me.
Sure, there are ass-hats, but the vast bulk of the time Apple is drowning in good input, and has to triage. The example listed here probably affects 1,000 to 2,000 developers, and a critical bug probably has to affect 150,000 people to get traction.
or you can spend $99 and prioritize your problem.
I had this problem with Microsoft and X509 about 15-20 years ago, (selection of which client certificate to identify a user when there is a certificate subjectName collision) -There was precisely one expert able to fix it, I was ringfenced from them, talking to a consultant and I had the great joy of hearing how much it would cost him to even TRY to get their attention on this problem, which btw, affects the US Government, so its not like it doesn't have "scale" behind it.
This is a problem you get, if you have hundreds of millions of customers.