Why do you think the problem you are talking about is specific to you and not for everyone?
Without more details, the described problem looks like a common problem for everyone: most people struggle to formalize their thoughts (and programming is a formalization). No one can translate their knowledge into code without mistakes. People make mistakes; it is okay.
Are you comparing yourself to someone else? Or to some “ideal” image of “how you should work”?
Maybe you could provide more concrete examples of your problem, like “I need to do X, but I struggle here and here and make mistakes here and here.”
“Personal” in “Describing myself without fitting in the possible category” fashion. Not really mean that I’m atypical, in fact, I do wish my problems were typical to an extent so that I would have credible guides in resolving this problem.
I’ll have an example that I personally experienced recently, omitting some details related to my PII.
Coding: I’m relatively familiar with data processing using Python. Reading from Text/JSON/Spreadsheet, sanitizing, running query and analyze (using modules and sometimes even interacting with scaffold NN models), formatting results. Normally not over 300 lines, mostly just piecing sample codes gathered from documents, other projects, and IntelliSense. But when a temporary analysis pipeline becomes a production practice, I’m required to upgrade the whole thing to the production level: Procedural must be wrapped with OOP abstraction for calling from other parts; Sanitazion designed for static data becomes generic for various inputs; Procedure shall be equipped thread-safe and concurrency-safe; etc., which requires vast rewrite and even if I manage to achieve some of the goals, the change would often render other change unusable.
I would like to write library-level projects in a modern and well-designed fashion, and my daily tasks often contain the kind of job that would make this kind of perfection worthy, not just worthy, but even critical. But as I stated in the question body, I’m not able to orchestrate them all together.
>I’ll have an example that I personally experienced recently, omitting some details related to my PII.
Based on this example, could you compare your expected outcome with the real outcome? In numbers. For example, “I expected to finish this refactoring in two weeks, but it took two months” or “I expected to finish this refactoring in two weeks, but my colleague completed the same task in two days.” Something like that.
I have more than 15 years of experience in software development, not precisely in data processing but in gamedev and backend (and sometimes in data processing), which is pretty close.
From my experience, most people are struggling with the task you described. Even senior developers with 5-10 years of experience. I saw only a few people, who could do such refactoring naturally and from the first try.
I may suggest a few things:
- Try to reevaluate your expectations. Maybe you are too hard on yourself. You may overestimate the required result and try to do more than you need. - Research a few methodologies of development. I may suggest Test Driven Development and Pair Programming. The first may help you control changes easier, and the second may help you learn how other people solve similar tasks. - Try to pay more attention to code decomposition and structuring when you work on “pipeline prototype”. Aka, add some “fake” functions/objects that will be required later, but for now, do nothing or contain trivial logic. Sometimes, it is easier to see what you need to do when you are fully in the task’s context rather than when you are refactoring your code. - Do not try to implement all features at the same time. Choose one, implement it, cover it with some tests or types or both, and then move to another.
Hi!
Why do you think the problem you are talking about is specific to you and not for everyone?
Without more details, the described problem looks like a common problem for everyone: most people struggle to formalize their thoughts (and programming is a formalization). No one can translate their knowledge into code without mistakes. People make mistakes; it is okay.
Are you comparing yourself to someone else? Or to some “ideal” image of “how you should work”?
Maybe you could provide more concrete examples of your problem, like “I need to do X, but I struggle here and here and make mistakes here and here.”
“Personal” in “Describing myself without fitting in the possible category” fashion. Not really mean that I’m atypical, in fact, I do wish my problems were typical to an extent so that I would have credible guides in resolving this problem.
I’ll have an example that I personally experienced recently, omitting some details related to my PII.
Coding: I’m relatively familiar with data processing using Python. Reading from Text/JSON/Spreadsheet, sanitizing, running query and analyze (using modules and sometimes even interacting with scaffold NN models), formatting results. Normally not over 300 lines, mostly just piecing sample codes gathered from documents, other projects, and IntelliSense. But when a temporary analysis pipeline becomes a production practice, I’m required to upgrade the whole thing to the production level: Procedural must be wrapped with OOP abstraction for calling from other parts; Sanitazion designed for static data becomes generic for various inputs; Procedure shall be equipped thread-safe and concurrency-safe; etc., which requires vast rewrite and even if I manage to achieve some of the goals, the change would often render other change unusable.
I would like to write library-level projects in a modern and well-designed fashion, and my daily tasks often contain the kind of job that would make this kind of perfection worthy, not just worthy, but even critical. But as I stated in the question body, I’m not able to orchestrate them all together.
>I’ll have an example that I personally experienced recently, omitting some details related to my PII.
Based on this example, could you compare your expected outcome with the real outcome? In numbers. For example, “I expected to finish this refactoring in two weeks, but it took two months” or “I expected to finish this refactoring in two weeks, but my colleague completed the same task in two days.” Something like that.
I have more than 15 years of experience in software development, not precisely in data processing but in gamedev and backend (and sometimes in data processing), which is pretty close.
From my experience, most people are struggling with the task you described. Even senior developers with 5-10 years of experience. I saw only a few people, who could do such refactoring naturally and from the first try.
I may suggest a few things:
- Try to reevaluate your expectations. Maybe you are too hard on yourself. You may overestimate the required result and try to do more than you need.
- Research a few methodologies of development. I may suggest Test Driven Development and Pair Programming. The first may help you control changes easier, and the second may help you learn how other people solve similar tasks.
- Try to pay more attention to code decomposition and structuring when you work on “pipeline prototype”. Aka, add some “fake” functions/objects that will be required later, but for now, do nothing or contain trivial logic. Sometimes, it is easier to see what you need to do when you are fully in the task’s context rather than when you are refactoring your code.
- Do not try to implement all features at the same time. Choose one, implement it, cover it with some tests or types or both, and then move to another.