I would say it’s a combination of being at the wrong company, and our education system being inadequate to the task.
There are many skills that are required in order to write complex software. You need to know how to organize your code in a maintainable and comprehensible way (Design Patterns, build/package systems, abstraction layers, even simple stuff like UML). You need to know how to find bugs in one’s own code as well as in code written by other people (using debuggers, reading stack traces, writing logs, applying basic deductive reasoning). When you get stuck, you need to know how to get help efficiently (reading documentation, understanding the jargon, knowing exactly which questions to ask, knowing whom to ask them to).
None of these skills are considered “sexy”; and, in fact, most scientists and mathematicians that I’ve worked with in the past don’t even recognize them as skills at all. Their attitude usually is, “don’t bother me with your bureaucratic design pattern bullshit, I wrote a 3000-line method that calculates an MDS plot and it works, what more do you want”. But the problem is that, without such skills, you will never be able to create anything more than a quick one-off script that performs one specific calculation and then quits.
My advice would be as follows.
Firstly, figure out what you actually want to do. Do you want to invent algorithms for other people to implement, or do you want to write software yourself ? There’s nothing wrong with either choice, but you need to consciously make the choice to begin with.
Secondly, if you do want to learn software engineering, find some people at your company who are already experienced software engineers. Ask them for a list of books or online tutorials to read (most likely, they’ll recommend the Design Patterns book, so you might as well start with that). After reading (or, let’s be realistic here, skimming) the books, ask them to sit down with you for a couple of hours in order to review your code—even, and especially, the code that actually works. Listen to their input, and refactor your code according to their recommendations. When you have a bug, make sure you’ve tried everything you could think of, and then ask them to sit down with you and walk you through the steps of diagnosing it.
Thirdly, if there are no such people at your current company, or if they flat-out refuse to help you… then find a better company :-(
I would say it’s a combination of being at the wrong company, and our education system being inadequate to the task.
There are many skills that are required in order to write complex software. You need to know how to organize your code in a maintainable and comprehensible way (Design Patterns, build/package systems, abstraction layers, even simple stuff like UML). You need to know how to find bugs in one’s own code as well as in code written by other people (using debuggers, reading stack traces, writing logs, applying basic deductive reasoning). When you get stuck, you need to know how to get help efficiently (reading documentation, understanding the jargon, knowing exactly which questions to ask, knowing whom to ask them to).
None of these skills are considered “sexy”; and, in fact, most scientists and mathematicians that I’ve worked with in the past don’t even recognize them as skills at all. Their attitude usually is, “don’t bother me with your bureaucratic design pattern bullshit, I wrote a 3000-line method that calculates an MDS plot and it works, what more do you want”. But the problem is that, without such skills, you will never be able to create anything more than a quick one-off script that performs one specific calculation and then quits.
My advice would be as follows.
Firstly, figure out what you actually want to do. Do you want to invent algorithms for other people to implement, or do you want to write software yourself ? There’s nothing wrong with either choice, but you need to consciously make the choice to begin with.
Secondly, if you do want to learn software engineering, find some people at your company who are already experienced software engineers. Ask them for a list of books or online tutorials to read (most likely, they’ll recommend the Design Patterns book, so you might as well start with that). After reading (or, let’s be realistic here, skimming) the books, ask them to sit down with you for a couple of hours in order to review your code—even, and especially, the code that actually works. Listen to their input, and refactor your code according to their recommendations. When you have a bug, make sure you’ve tried everything you could think of, and then ask them to sit down with you and walk you through the steps of diagnosing it.
Thirdly, if there are no such people at your current company, or if they flat-out refuse to help you… then find a better company :-(