These are all excellent questions.

> What do you do when you don't have a spec and when you ask the senior engineer you are working with questions, they don't respond or push back against having a spec or a plan longer than the very next feature?

Oh man, it drives me nuts to hear this. I'm a senior engineer, and you know what my biggest complaint is about many junior engineers? They don't ask enough questions! So to hear that you're asking questions and not getting a helpful response is beyond frustrating.

And in my view, a senior engineer who won't plan beyond the next feature is incompetent. I mean, I sometimes am forced to admit that I haven't figured something out yet, or in some cases haven't even thought about it yet, but if it's important, I make a point to figure it out. Maintaining architectural coherence of the product in the long term is, in my view, one of my most important jobs.

> The author says that estimation is easier to do if you are doing something you have done before. What do you do if you are always working with a new framework or language?

> How do you break down the steps if you don't know what a project entails until you are finished?

I hope you're not really _always_ working with a new framework or language, but yes, in software we're usually doing something new for every project, because it would be pointless to do the same thing over again.

You'll have to take anything I say about estimation with a grain of salt, because I've been in the business 35 years and I still suck at estimation. But your second question hints at the best answer I know, which is that you have to get well started on a project -- at least 1/4 of the way in -- before you can begin to give confident estimates of how long it will take.

What the OP is suggesting is to try to go at it breadth-first rather than depth-first: figure out what the top-level tasks are, then their immediate subtasks, then the level below that, etc., rather than just diving into the first thing until you get it done. I'm sure they have a point and I don't do enough of this. But the counterpoint is that doing this is a task in itself, and takes time -- to do it well takes a significant fraction of the time of the project. If the business needs are such that that makes sense, then do it.

> What do you do if asked to estimate how long something will take to debug?

You say "I don't know." If you've tracked down other bugs in the same code before, you can offer that in the past, bugs in that code have taken so many hours or days, but you can't promise this one won't be harder. If you have no experience with that code, though, stick to saying you don't know.

> I've gotten the feedback that I should avoid trying to "understand the universe" when debugging.

Ay ay ay. Find a new job.

The last thing I want is my team members making fixes without at least trying to fully understand the consequences. Sometimes we think we understand but it turns out that we didn't, and that's life, but not to even make the attempt is a sure path to code rot... making the code even harder to understand for the next person. I guess some people get inured to the idea that they'll never fully understand what's going on and they have to try to fix it anyway, but in my opinion, the best engineers always try hard to understand first.

> Why do experienced engineers ask for estimates anyway?

Well, in fairness, there are cases where we have multiple things to fix, and their relative priority depends partly on how much time they're likely to take. So getting some kind of order-of-magnitude estimate, if possible, can be helpful. But how experienced engineers can forget that there is always an element of guesswork in these is beyond me.

> How do I deal with it when they are upset about it taking a week and a half?

If they won't give you walkthroughs, and they won't answer questions, and they still get upset when things take time... yeah, get a new job.

For a lot of these, rather than trying to estimate the thing you're asked to estimate, you need to either estimate how long it'll take you to do the analysis you need in order to come up with an estimate, or you need to provide a time-limit on the task after which you'll provide a status update and have a discussion about the next step.

Managers don't really care about time, they care about budgets and risks. In your debugging example, the existence of the bug is a big unknown risk, and your manager needs to manage that. You obviously can't tell how long it'll take to fix the bug, but you can say "Give me an hour to debug it, and after that I should have a better idea of what the cause is and how long it'll take to fix. I might even have it fixed already by that point." That gives your manager a budget (one hour) and risk control (more information in a fixed time), and the option after a small investment to bring someone else in to assist.

It sounds like you are saying that when I take an hour to work on debugging something, I should spend 55 minutes taking notes on the system and where the problem might be and 5 minutes turning those notes into a coherent email. Is that a reasonable approach?

I certainly prefer to think of debugging as a process of learning about a system rather that trying to try things at random without making a mental map of where I am.

I'm saying that when asked for a debugging estimate, you should commit to spending no more than an hour (or other amount, depending on the problem) before stopping to report on your progress. How you spend that hour depends on the circumstances; if you could predict that then you probably already know enough to give a somewhat accurate estimate.

When you stop to report on your progress, you don't need to deliver detailed notes. It could be as simple as "yeah, fixed it" or "figured out the problem, I need three hours to refactor the code to resolve it" or "based on what I'm seeing, Bob would be better at fixing this; I can spend a few minutes to show him how to reproduce it."

"try things at random" is not debugging, so you're right about the process.

usually when I debug things my method is sort of a mix of linear and binary search where I start at one place, see if the a value is what I expect it to be, then move along to a different part of the code to see if something is wrong there, reducing the search space and repeating as necessary.

Sometimes my method is instead to look at data output at a certain level of abstraction like system calls (which is usually not helpful except for detecting a network hang or a path config whose directory doesn't exist) or database queries or something. That is usually more helpful if I either already suspect I know where the problem is and need to know what model name or file name to `ag` for.

If (Hephaestus be praised) there is actually an automated test framework in place, I'll start by writing a failing test or copy-pasting an existing test and modifying it to fail, then (if it isn't JavaScript), inserting a debugger into a function the test calls.

All of this tends to be pretty successful.

But I've occasionally found myself on a project where I just have a sense that I don't know what is going on or how pieces fit together and I feel a strong temptation to stop and fix that sense of bewilderment. When I resist this urge is when it feels like I am trying things at random.

It comes down to experience really. The more you are exposed to, if you are paying attention, the easier it becomes to relate to previous events, which can help inform estimates.

If you don't have a spec, get a spec. A Senior engineer may be more reluctant to help you with implementation direction without knowing the requirements.

Sometimes you need to do research to give an estimate. If someone asks you in person, or over e-mail, and you honestly don't know ask them if you can get back to them. Do not give them a number. Do the research, then reply.

If you're relying on external APIs or Frameworks etc. You need to factor in Risk Assessments that include what happens if the fit is wrong, or the interface changes mid project, etc.

When it comes to debugging a lot comes into play, system design, documentation, experience with the code. "I don't know" is an acceptable answer until you understand what is going on. Once you understand what is going on, you then estimate the fix based on the code. Giving a number before you understand what you are fixing in detail is a bad plan.

I am exceptionally experienced with a certain code base and system. You may be experienced with a different code base. You're more qualified to handle estimation on what you know. That said, if you honestly don't know, say that. Don't low ball the amount of work if you actually have no idea how long it will take.

There are a few good books on estimation and debugging, look around maybe they'll give you a few ideas.

> comes down to experience.

Yea. I really should get into the habit of keeping better records of things so that I can look back on past projects. I write in a paper notebook as I program, but that is usually just thinking out loud or trying to artificially increase my working memory.

> If you don't have a spec, get a spec.

Yea, I think I've concluded that I need to always write a spec for myself before I start a project. I know Joel says that PMs should write specs, but I should be prepared for the reality that projects rarely have separate PMs.

The hard part is how to write out the technical details ahead of time before I've gotten my mind into the codebase Im adding onto or the new framework I'm using.

> Risk assessments

Any tips on what to read to do these effectively?

> Once you understand what is going on

So I should feel safe in disregarding the advice to "avoid trying to understand the universe"? Or at least attribute it more to how I communicate about being stuck on a problem? That is kinda a relief. I really didn't understand how to put that advice into practice.

> A few good books on estimation and debugging

I've read Debugging: 9 Indespensible Rules, Mythical Man-Month, Critical Chain, and a few chapters from Software Estimation: Demystifying the Black Art. Do you have any other recommendations? The latter three books seem to be talking on the level of projects that are anticipated to take at least 6 months. I'm looking to start out with being able to reliably estimate how long I can get done in 90 minutes.

> Do you have any other recommendations?

You might take a look at www.amazon.com/Rapid-Development-Taming-Software-Schedules/dp/1556159005/. Although it's 20 years old, it has some good ideas in it. It's also more about 6-month estimations than 90-minute estimations, though.

The Pragmatic Programmer: From Journeyman to Master is a good book I always recommend to developers I'm mentoring.

I read that as a freshman in college. I should probably re-read it though. It seemed kinda like How to Win Friends and Influence People.

> What do you do when you don't have a spec and when you ask the senior engineer you are working with questions, they don't respond or push back against having a spec or a plan longer than the very next feature?

What I would suggest is that you should learn the practices of wherever you are working, and not spend much effort trying to change them. Your first couple of jobs, you'll hopefully be exposed to several different methodologies, and only then will you have the experience to advocate for best practices, as well as being able then to speak to evidence of it working at a past job.

> What do you do if asked to estimate how long something will take to debug? What if you just started on a project and the error message you are trying to debug doesn't make sense?

If you can't accurately estimate debugging, instead explain what you understand, what you don't understand, what you've learned so far, and what you're planning on trying next.

Well I'm currently on the job hunt due to failure to deliver on projects at my most recent job, so I'm trying to figure out what to both change about my practices and what to look for in the practices at the place where I work next.

As a general rule, if management expects results without giving you the tools for the job, ask yourself whether you really want to be in that job, and start looking for alternatives. That said, there's several ways you can make things better:

> What do you do when you don't have a spec and when you ask the senior engineer you are working with questions, they don't respond or push back against having a spec or a plan longer than the very next feature?

Illustrate why you need the answer. "I'm trying to decide between approach A or approach B. If we're doing X then approach A is easier to implement, but that'll suck if we're doing Y, in which case the extra work involved in B is worthwhile".

> The author says that estimation is easier to do if you are doing something you have done before. What do you do if you are always working with a new framework or language?

If you have experience with Express in Node.js, Ruby + Sinatra or Python + Flask will seem familiar to you. Rails or Django not so much. C++ would be more alien to you than Python or Ruby. Learn to identify such similarities and differences. I tend to find that domain knowledge is harder to come by than language/library knowledge -- The exact details of the work might be different, but if you're used to writing web apps or compilers or games or whatever, then you should be able to ballpark estimate such a project in a different language. Larger projects are easier too -- the cost for picking up the tools is roughly fixed and gets amortised over the whole lifetime of the project.

> - How do you break down the steps if you don't know what a project entails until you are finished?

First off: If we're talking about more than a couple of days' worth of work, estimating a project isn't a 5-minute job. Think a few hours for a 2-3 week project, at least, if you want a precise estimate. And there will be things you miss, but that's why you always add some padding.

At a minimum, you need to ask yourself the following questions: You're writing a service that does X. How does it receive input from the users? How does it pass output to the users? How does it obtain the data necessary to do X? Once you have the user input and the data, what work is involved in performing the task X?

Note how all these questions all apply irrespectively of whether you're writing a web service, a compiler or a game (with the details of what they _mean_ being quite different). You spend some time researching potential answers, and come up with a sketch of what an implementation looks like. Then you iterate: for each piece of the puzzle, what's involved. One of the easiest rookie mistakes to make here is to account for the time necessary to _implement_ all the pieces (and often that will be accurate), but allow remarkably little to no time for integrating the pieces together, for testing, for documentation, and for all other such things.

> What do you do if asked to estimate how long something will take to debug?

Tell them to f* off, in the most abrasive tone possible that is appropriate for your relationship with the other party. Seriously though, estimating how long it takes to sort out a bug is much, much harder than estimating new development. Time-to-fix-a-bug is also a long tail sort of phenomenon, so be prepared for that.

> This excercise takes a lot of time. What do you do when asked for an estimate in a meeting rather than over email?

After some projects you'll soon start being able to come up with a few good questions that need answering before you can give a good estimate. Bring them up, say you need to think about it, and be firm.

> - What do you do if the thing you are building relies on getting an external API to work and that API is either undocumented or is documented in a foreign language? Assume that the google translation is not making sense and this is your first project and you don't have a budget to hire a professional translator? This sounds like something you would need to get working before you could actually give an estimate.

Estimate the estimation. Say you'll need some time to figure out the requirements before you can safely say what work is involved.

> Why do experienced engineers ask for estimates anyway? Every time I give one I feel like I am lying and I try to warn people "I really don't know how to come up with estimates but I am guessing 3 hours". How do I deal with it when they are upset about it taking a week and a half?

This is probably the single most important part, and most places I've seen fuck it up tremendously. It's crucially important that people get to talk about this without turning it into a blame game. Why did you think it would take three hours, and why did it take a week and a half instead? Clearly either the estimation was wrong, or something _really_ bizarre happened that made the estimate invalid. Estimation is a skill that you need to learn, and you'll need to learn it before you're any good at it. My experience is that you _can_ estimate things correctly (down to the half-day on projects lasting multiple weeks), once you have the context to do it in.

> As a general rule, if management expects results without giving you the tools for the job, ask yourself whether you really want to be in that job, and start looking for alternatives.

Well the hard part here is determining if lack the tools due to management disorganization or due to my own lack of skill or intelligence. I have already left the organization.

> you _can_ estimate things correctly once you have the context

Maybe my problem comes down to not recognising when I lack a piece of context or being stubborn enough to get it.

You've been given a lot of good advice on here - I concur with all of it. After almost 20 years of writing code I still couldn't tell you how long it would take to debug something in my own codebase. If I can visualise it, sure, I can take a guess, but if not, who knows what I'm going to find.

Given the questions you've asked on here, it really sounds like you were just in a bad environment. You said 3 hours, it took a week - you're a junior, why the hell didn't someone jump in to help out?

Everything you've asked comes down to experience and you need people around to support you in gaining that experience.

Good luck finding a nicer workplace - honestly, sounds like you have the makings of a good inquisitive developer mind. There are plenty of places out there that will help you develop your full potential.