I’m a recent grad who’s spent the last six years formally studying mathematics and informally learning programming. I have experience writing code for school projects and I did a brief but very successful math-related internship that involved coding. I was a high-performing student in mathematics and I always thought I was good at coding too, especially back in high school when I did programming contests and impressive-for-my-age personal projects.
A couple months ago I decided to look for a full-time programming job and got hired fairly soon, but since then it’s been a disaster. I’m at a fast-moving startup where I need to learn a whole constellation of codebase components, languages, technologies, and third-party libraries/frameworks but I’m given no dedicated time to do so. I was immediately assigned a list of bugs to fix and without context and understanding of the relevant background knowledge I frantically debug/google/ask for help until somehow I discover the subtle cause of the bug. Three times already I’ve received performance pressure, and things aren’t necessarily looking up. Other new hires from various backgrounds seem to be doing just fine. All this despite my being a good coder and a smart person even by LW standards. I did well in the job interview.
When I was studying and working in academia, I found that the best way to be productive at something (say, graph theory research) is to gradually transition from learning background to producing output. Thoroughly learning background in an area is an investment with great returns since it gives me context and a “top-down” view that allows me to quickly answer questions, place new knowledge into an already dense semantic web, and easily gauge the difficulty of a task. I could attempt to go into more details but the core is: Based on my experience, “hitting the ground running” by prioritizing quick output and only learning background knowledge as necessary per task is inefficient and I’m bad at it.
At the moment my only strong technology skills are the understanding of the syntax and semantics of a couple of programming languages.
Am I at the wrong company? Am I in the wrong profession—should I go back to academia, spend four years getting a PhD, and work in more mathy positions? Thanks!
I would say it’s a combination of being at the wrong company, and our education system being inadequate to the task.
There are many skills that are required in order to write complex software. You need to know how to organize your code in a maintainable and comprehensible way (Design Patterns, build/package systems, abstraction layers, even simple stuff like UML). You need to know how to find bugs in one’s own code as well as in code written by other people (using debuggers, reading stack traces, writing logs, applying basic deductive reasoning). When you get stuck, you need to know how to get help efficiently (reading documentation, understanding the jargon, knowing exactly which questions to ask, knowing whom to ask them to).
None of these skills are considered “sexy”; and, in fact, most scientists and mathematicians that I’ve worked with in the past don’t even recognize them as skills at all. Their attitude usually is, “don’t bother me with your bureaucratic design pattern bullshit, I wrote a 3000-line method that calculates an MDS plot and it works, what more do you want”. But the problem is that, without such skills, you will never be able to create anything more than a quick one-off script that performs one specific calculation and then quits.
My advice would be as follows.
Firstly, figure out what you actually want to do. Do you want to invent algorithms for other people to implement, or do you want to write software yourself ? There’s nothing wrong with either choice, but you need to consciously make the choice to begin with.
Secondly, if you do want to learn software engineering, find some people at your company who are already experienced software engineers. Ask them for a list of books or online tutorials to read (most likely, they’ll recommend the Design Patterns book, so you might as well start with that). After reading (or, let’s be realistic here, skimming) the books, ask them to sit down with you for a couple of hours in order to review your code—even, and especially, the code that actually works. Listen to their input, and refactor your code according to their recommendations. When you have a bug, make sure you’ve tried everything you could think of, and then ask them to sit down with you and walk you through the steps of diagnosing it.
Thirdly, if there are no such people at your current company, or if they flat-out refuse to help you… then find a better company :-(
This is very much from the outside, but how sure are you that the other new hires are doing just fine? Could they (or some of them) be struggling like you are?
When I started as a programmer I joined a graduate program at a big company. I was also fortunate enough that one of the consultants working there was able to act as an informal mentor in how things are done in “real world” programming (including dealing with all those technologies, frameworks, etc). You might find it easier to get up to speed with a bigger, slower-moving company with a more long-term view than a frantic startup.
One framing that might be useful: Part of being a professional software engineer is learning new things constantly, whether it’s new languages, new frameworks, new codebases, etc. But this itself is a skill that can be learned and practiced. In addition to having the attitude of relentless resourcefulness, there are many small tricks that can be picked up: for example, grepping for a bit of text from the UI to quickly find the code that defines it, using Google’s site: operator to make it easier to do targeted documentation searches, using your language’s debugger to solve bugs, having an editor with lots of plugins installed that make you more efficient, etc. This sentence: “I was immediately assigned a list of bugs to fix and without context and understanding of the relevant background knowledge I frantically debug/google/ask for help until somehow I discover the subtle cause of the bug” sounds like a pretty good description of what I found it was like to work as a software engineer at a startup, minus the word “frantic”. I spent a lot of time learning just enough about the code I was working with to solve the problem I needed to, searching on google for just enough documentation to accomplish what I needed to accomplish, etc. Even the best software developers are doing keyword searches on Google and their codebase constantly as part of their development flow; if you find yourself doing this you should not consider it indicative of a problem.
Based on what you describe as your programming background, it sounds like you don’t have much experience with this modality of software development. Probably lots of other recent hires have experience as interns or teaching themselves stuff for independent projects, which helped them learn the skill of just-in-time knowledge acquisition. I might try working at a less demanding company just so you could feel less stressed out and give yourself the opportunity to gradually ramp up to this development style. If you work at a bigger, slower-moving company that nevertheless is using fairly up-to-date technologies and you spend 15 minutes every morning working to improve your tools & efficiency, my guess is you’ll be in a much better place after a year or so.
Seems to me that “good job” is a 2-place word—which working environments makes which people productive? I have experiences very similar to what you described (dozen new components, zero time to learn, expected high productivity immediately, other people seem to cope well and receive performance bonuses). But I also have experiences of high productivity in business situations, where I received performance bonuses. Sometimes both experiences in the same company, just in a different project or with a different boss.
My “natural” approach is to do some learning first: I try to make sure that I understand the specification, that it is complete and unambiguous. For example I will first sketch the dialogs on paper and ask my boss or customer whether this is what they meant (because redrawing a sketch is much easier, faster, cheaper, and less frustrating than changing the code of an already developed and tested program). I think about all the additional work which was not mentioned but may be logically necessary, and I ask about it in advance, offering the least complicated solution. (“You want to have a list of users with passwords. What happens if someone forgets their password? Perhaps there could be an administrator who can change passwords, and as an almost free bonus, they could also block and unblock users. Or maybe users could provide their e-mails, and then change their passwords using e-mail verification, but that would be more complicated, and some users will call your support anyway, so unless you have thousands of users I believe this is not necessary, and can still be included later.”) I also research the framework I am supposed to use, and make a few simple prototypes with the functionality I expect to need in the project. (Thus, if the framework has some serious problems, I am likely to find out in advance, when there is still a chance to use a different technology, or at least to avoid the problematic parts of the framework. The fact that I am not under pressure yet helps me notice more details and context.) Then, when I am ready (which is never perfect, but that’s not the point), I implement the solution, step by step. At each moment I know where I am in the project and how much needs to be done yet. (This knowledge usually also makes my boss happy, assuming they believe my analysis.) I do a part, and I test it. When I am ready, there are usually very few bugs. So the time “wasted” analysing the problem and doing the prototypes is later offset by less bug fixing and less remakes. And there is generally less stress.
As an example of my work (probably the only example available freely online), I did a cycle route map for Trnava region in one month; and that included learning the Google Maps framework which I have never used before. A short tour: Choosing “Zoznam cyklotrás” displays a list of cycle routes; clicking on one of them displays it on the map. The “Zobrazenie cesty k bodom mimo trasy” checkbox allows you to click anywhere on the map to display the shortest path from that place to the route. There is also a possibility to find a cycle route according to your criteria, display “places of interest” within given distance from the route; your results can be printed or exported to PDF. This is the part visible to the user. The administrator part allows importing the cycle routes from GPS log files, editing and annotating the route points (e.g. “crossing a road”, “crossing a railway”); and importing the “places of interest” from an Excel file. -- I believe I was rather productive here, but of course I am open to feedback. (I am not working for that company anymore, so I can’t use the feedback to improve the product.)
However, most managers insist on a completely different approach to work, which other programmers seem to handle somehow (some enjoy it, others complain but cope anyway), but it makes me suffer and less productive. I would describe it as chaotic, short-sighted, penny-wise and pound-foolish, treating experts as replaceable, and accumulating the technical debt until the whole thing collapses under its own weight. Learning the new technology or debating the details of the project is considered an unproductive waste of time; the important thing is keeping the programmers busy (even if it means doing something merely to tear it down later). Maybe it’s because the managers cannot recognize productivity, but can recognize silence and typing. And I really hate the idea that people are completely replaceable and should be randomly switched between different parts of the project or even random projects; in real life it often means the code is sloppy and undocumented, and there is no time or incentive to fix it, because if you spend your time refactoring the code, it makes you seem less productive, and the benefits will be enjoyed by someone else when you are moved to the next piece of spaghetti code. (Somehow collective code ownership is the part of agile programming most accepted by self-labeled enlightened managers. The other parts like unit testing or pair programming are obviously just a waste of time. I have to ask Robin Hanson whether it is a coincidence that the managers embrace exactly the one part which allows them to reduce the perceived status of expert programmers.) Well, to be fair, sometimes there are also external constraints, such as the customer insisting on doing things the stupid way; but I believe most of this comes from inside of the companies.
Most of my productive work was done when I worked on a project alone; and once when I was a leader of a small project. My approach as the leader was asking people what are their individual strengths and what part of the project would they prefer to do, and dividing the work accordingly. Then I paid attention to have clean APIs between the team members, good documentation of the data, and sketches of the dialogs. And then we just coded, and sometimes debated. Three programmers, myself included, the other two were part-time working students, for me it was the first Java EE project done from the scratch and I didn’t have experience with many parts. Yet we completed it in three months, and received bonuses.
Programming with other people, working with large codebases and working with multiple libraries and frameworks are basically all software engineering realities that education gives minimal training for. If you can view the job as a learning experience, it’s probably a good one even though it is frustrating on multiple levels right now. If you’re scrupulous about needing to pull the same weight as the other members and are thinking about switching jobs because of this, you could just talk about it to your manager. They might concede that yeah, you’re not a good fit, and then you can go find a less miserable place to work in, or say that they think you’re actually doing fine, in which case you can go back to considering the learning experience perspective.
Can’t advise on the taking the time off to do a PhD instead, but I don’t think you should give up on programming just yet. Like others said, there are many companies, and bigger companies have more resources to spend for training and mentoring. Also, the current mode of mixing together a bunch of frameworks developed in the last few years nobody really understands and rushing to the market with a minimum viable product chock-full of technical debt is probably just an artifact of the web as an application platform still being a reasonably new thing and people rushing to figure out all the simple things they can do with it. If the technology stabilizes, there’s going to be more opportunity for mastering long-lived technologies.
On the other hand, becoming a specific technology expert in programming is a gamble. Technologies just plain up die sometimes. Math domain expertise is probably a lot more durable, but it’s probably also trickier to get a nice math job than it is to get a nice programming job.
Looks like a wrong company. Try looking for something more structured. It’s good to learn the proper requirements/design/coding/testing techniques before throwing them out the window.
(request for guidance from software engineers)
I’m a recent grad who’s spent the last six years formally studying mathematics and informally learning programming. I have experience writing code for school projects and I did a brief but very successful math-related internship that involved coding. I was a high-performing student in mathematics and I always thought I was good at coding too, especially back in high school when I did programming contests and impressive-for-my-age personal projects.
A couple months ago I decided to look for a full-time programming job and got hired fairly soon, but since then it’s been a disaster. I’m at a fast-moving startup where I need to learn a whole constellation of codebase components, languages, technologies, and third-party libraries/frameworks but I’m given no dedicated time to do so. I was immediately assigned a list of bugs to fix and without context and understanding of the relevant background knowledge I frantically debug/google/ask for help until somehow I discover the subtle cause of the bug. Three times already I’ve received performance pressure, and things aren’t necessarily looking up. Other new hires from various backgrounds seem to be doing just fine. All this despite my being a good coder and a smart person even by LW standards. I did well in the job interview.
When I was studying and working in academia, I found that the best way to be productive at something (say, graph theory research) is to gradually transition from learning background to producing output. Thoroughly learning background in an area is an investment with great returns since it gives me context and a “top-down” view that allows me to quickly answer questions, place new knowledge into an already dense semantic web, and easily gauge the difficulty of a task. I could attempt to go into more details but the core is: Based on my experience, “hitting the ground running” by prioritizing quick output and only learning background knowledge as necessary per task is inefficient and I’m bad at it.
At the moment my only strong technology skills are the understanding of the syntax and semantics of a couple of programming languages.
Am I at the wrong company? Am I in the wrong profession—should I go back to academia, spend four years getting a PhD, and work in more mathy positions? Thanks!
I would say it’s a combination of being at the wrong company, and our education system being inadequate to the task.
There are many skills that are required in order to write complex software. You need to know how to organize your code in a maintainable and comprehensible way (Design Patterns, build/package systems, abstraction layers, even simple stuff like UML). You need to know how to find bugs in one’s own code as well as in code written by other people (using debuggers, reading stack traces, writing logs, applying basic deductive reasoning). When you get stuck, you need to know how to get help efficiently (reading documentation, understanding the jargon, knowing exactly which questions to ask, knowing whom to ask them to).
None of these skills are considered “sexy”; and, in fact, most scientists and mathematicians that I’ve worked with in the past don’t even recognize them as skills at all. Their attitude usually is, “don’t bother me with your bureaucratic design pattern bullshit, I wrote a 3000-line method that calculates an MDS plot and it works, what more do you want”. But the problem is that, without such skills, you will never be able to create anything more than a quick one-off script that performs one specific calculation and then quits.
My advice would be as follows.
Firstly, figure out what you actually want to do. Do you want to invent algorithms for other people to implement, or do you want to write software yourself ? There’s nothing wrong with either choice, but you need to consciously make the choice to begin with.
Secondly, if you do want to learn software engineering, find some people at your company who are already experienced software engineers. Ask them for a list of books or online tutorials to read (most likely, they’ll recommend the Design Patterns book, so you might as well start with that). After reading (or, let’s be realistic here, skimming) the books, ask them to sit down with you for a couple of hours in order to review your code—even, and especially, the code that actually works. Listen to their input, and refactor your code according to their recommendations. When you have a bug, make sure you’ve tried everything you could think of, and then ask them to sit down with you and walk you through the steps of diagnosing it.
Thirdly, if there are no such people at your current company, or if they flat-out refuse to help you… then find a better company :-(
This is very much from the outside, but how sure are you that the other new hires are doing just fine? Could they (or some of them) be struggling like you are?
Thanks for the response. It’s hard to say exactly, but I can see their work logs and I hear them getting congratulated from time to time.
When I started as a programmer I joined a graduate program at a big company. I was also fortunate enough that one of the consultants working there was able to act as an informal mentor in how things are done in “real world” programming (including dealing with all those technologies, frameworks, etc). You might find it easier to get up to speed with a bigger, slower-moving company with a more long-term view than a frantic startup.
One framing that might be useful: Part of being a professional software engineer is learning new things constantly, whether it’s new languages, new frameworks, new codebases, etc. But this itself is a skill that can be learned and practiced. In addition to having the attitude of relentless resourcefulness, there are many small tricks that can be picked up: for example, grepping for a bit of text from the UI to quickly find the code that defines it, using Google’s site: operator to make it easier to do targeted documentation searches, using your language’s debugger to solve bugs, having an editor with lots of plugins installed that make you more efficient, etc. This sentence: “I was immediately assigned a list of bugs to fix and without context and understanding of the relevant background knowledge I frantically debug/google/ask for help until somehow I discover the subtle cause of the bug” sounds like a pretty good description of what I found it was like to work as a software engineer at a startup, minus the word “frantic”. I spent a lot of time learning just enough about the code I was working with to solve the problem I needed to, searching on google for just enough documentation to accomplish what I needed to accomplish, etc. Even the best software developers are doing keyword searches on Google and their codebase constantly as part of their development flow; if you find yourself doing this you should not consider it indicative of a problem.
Based on what you describe as your programming background, it sounds like you don’t have much experience with this modality of software development. Probably lots of other recent hires have experience as interns or teaching themselves stuff for independent projects, which helped them learn the skill of just-in-time knowledge acquisition. I might try working at a less demanding company just so you could feel less stressed out and give yourself the opportunity to gradually ramp up to this development style. If you work at a bigger, slower-moving company that nevertheless is using fairly up-to-date technologies and you spend 15 minutes every morning working to improve your tools & efficiency, my guess is you’ll be in a much better place after a year or so.
Seems to me that “good job” is a 2-place word—which working environments makes which people productive? I have experiences very similar to what you described (dozen new components, zero time to learn, expected high productivity immediately, other people seem to cope well and receive performance bonuses). But I also have experiences of high productivity in business situations, where I received performance bonuses. Sometimes both experiences in the same company, just in a different project or with a different boss.
My “natural” approach is to do some learning first: I try to make sure that I understand the specification, that it is complete and unambiguous. For example I will first sketch the dialogs on paper and ask my boss or customer whether this is what they meant (because redrawing a sketch is much easier, faster, cheaper, and less frustrating than changing the code of an already developed and tested program). I think about all the additional work which was not mentioned but may be logically necessary, and I ask about it in advance, offering the least complicated solution. (“You want to have a list of users with passwords. What happens if someone forgets their password? Perhaps there could be an administrator who can change passwords, and as an almost free bonus, they could also block and unblock users. Or maybe users could provide their e-mails, and then change their passwords using e-mail verification, but that would be more complicated, and some users will call your support anyway, so unless you have thousands of users I believe this is not necessary, and can still be included later.”) I also research the framework I am supposed to use, and make a few simple prototypes with the functionality I expect to need in the project. (Thus, if the framework has some serious problems, I am likely to find out in advance, when there is still a chance to use a different technology, or at least to avoid the problematic parts of the framework. The fact that I am not under pressure yet helps me notice more details and context.) Then, when I am ready (which is never perfect, but that’s not the point), I implement the solution, step by step. At each moment I know where I am in the project and how much needs to be done yet. (This knowledge usually also makes my boss happy, assuming they believe my analysis.) I do a part, and I test it. When I am ready, there are usually very few bugs. So the time “wasted” analysing the problem and doing the prototypes is later offset by less bug fixing and less remakes. And there is generally less stress.
As an example of my work (probably the only example available freely online), I did a cycle route map for Trnava region in one month; and that included learning the Google Maps framework which I have never used before. A short tour: Choosing “Zoznam cyklotrás” displays a list of cycle routes; clicking on one of them displays it on the map. The “Zobrazenie cesty k bodom mimo trasy” checkbox allows you to click anywhere on the map to display the shortest path from that place to the route. There is also a possibility to find a cycle route according to your criteria, display “places of interest” within given distance from the route; your results can be printed or exported to PDF. This is the part visible to the user. The administrator part allows importing the cycle routes from GPS log files, editing and annotating the route points (e.g. “crossing a road”, “crossing a railway”); and importing the “places of interest” from an Excel file. -- I believe I was rather productive here, but of course I am open to feedback. (I am not working for that company anymore, so I can’t use the feedback to improve the product.)
However, most managers insist on a completely different approach to work, which other programmers seem to handle somehow (some enjoy it, others complain but cope anyway), but it makes me suffer and less productive. I would describe it as chaotic, short-sighted, penny-wise and pound-foolish, treating experts as replaceable, and accumulating the technical debt until the whole thing collapses under its own weight. Learning the new technology or debating the details of the project is considered an unproductive waste of time; the important thing is keeping the programmers busy (even if it means doing something merely to tear it down later). Maybe it’s because the managers cannot recognize productivity, but can recognize silence and typing. And I really hate the idea that people are completely replaceable and should be randomly switched between different parts of the project or even random projects; in real life it often means the code is sloppy and undocumented, and there is no time or incentive to fix it, because if you spend your time refactoring the code, it makes you seem less productive, and the benefits will be enjoyed by someone else when you are moved to the next piece of spaghetti code. (Somehow collective code ownership is the part of agile programming most accepted by self-labeled enlightened managers. The other parts like unit testing or pair programming are obviously just a waste of time. I have to ask Robin Hanson whether it is a coincidence that the managers embrace exactly the one part which allows them to reduce the perceived status of expert programmers.) Well, to be fair, sometimes there are also external constraints, such as the customer insisting on doing things the stupid way; but I believe most of this comes from inside of the companies.
Most of my productive work was done when I worked on a project alone; and once when I was a leader of a small project. My approach as the leader was asking people what are their individual strengths and what part of the project would they prefer to do, and dividing the work accordingly. Then I paid attention to have clean APIs between the team members, good documentation of the data, and sketches of the dialogs. And then we just coded, and sometimes debated. Three programmers, myself included, the other two were part-time working students, for me it was the first Java EE project done from the scratch and I didn’t have experience with many parts. Yet we completed it in three months, and received bonuses.
Programming with other people, working with large codebases and working with multiple libraries and frameworks are basically all software engineering realities that education gives minimal training for. If you can view the job as a learning experience, it’s probably a good one even though it is frustrating on multiple levels right now. If you’re scrupulous about needing to pull the same weight as the other members and are thinking about switching jobs because of this, you could just talk about it to your manager. They might concede that yeah, you’re not a good fit, and then you can go find a less miserable place to work in, or say that they think you’re actually doing fine, in which case you can go back to considering the learning experience perspective.
Can’t advise on the taking the time off to do a PhD instead, but I don’t think you should give up on programming just yet. Like others said, there are many companies, and bigger companies have more resources to spend for training and mentoring. Also, the current mode of mixing together a bunch of frameworks developed in the last few years nobody really understands and rushing to the market with a minimum viable product chock-full of technical debt is probably just an artifact of the web as an application platform still being a reasonably new thing and people rushing to figure out all the simple things they can do with it. If the technology stabilizes, there’s going to be more opportunity for mastering long-lived technologies.
On the other hand, becoming a specific technology expert in programming is a gamble. Technologies just plain up die sometimes. Math domain expertise is probably a lot more durable, but it’s probably also trickier to get a nice math job than it is to get a nice programming job.
Looks like a wrong company. Try looking for something more structured. It’s good to learn the proper requirements/design/coding/testing techniques before throwing them out the window.