I don’t know anything about the politics of slavery in the US at the time, but in general, a relevant question is: how strongly did the non-slaveholders desire slavery to be outlawed, as compared to their desires with regard to other issues?
In general, in politics it’s quite common that the majority of the populace has a moderate preference to do X, a much smaller fraction of the populace has a very strong preference to not do X, and the desires of the minority win out. For the majority, the issue might not be important enough that they’d change their vote because of it, especially if the politician in question supports other issues who the people feel are more important. For the minority, however, the issue may be important enough to be the deciding factor in whom they vote. So the politican will maximize their votes by doing what the minority wants with regard to issue X, and what the majority wants with regard to everything else. At the same time, if the minority and majority vote for different politicians, then it’s beneficial for the elected politicians to barter votes, so that the majority “buys” the minority’s support for laws that they might not be able to pass otherwise, in exchange for giving the minority what they want on an issue that feels less important for the majority.
Of course, all of this presumes that the voters act “rationally”, and give their support to the candidates who most accurately match the desires of the voters. Pretty much everything that we know about voter behavior says that this isn’t the case. (Rationally was in scare quotes because, given how little influence a single vote actually has on an election, not bothering to figure out what your candidate actually does may in fact be the most rational use of your time.)
On a somewhat related note, non-slaveholders often bought goods that involved slave labor at some point in the process of their production. It’s quite possible that they at least thought that freeing the slaves would raise the cost of their tobacco, clothes ect.
edited to add
This is a bit more speculative, but I suspect that labor was significaly non-fungibal, and in particular that Northerner’s didn’t consider Southern agricultural jobs to be closely tied to their own labor market.
I don’t know anything about the politics of slavery in the US at the time, but in general, a relevant question is: how strongly did the non-slaveholders desire slavery to be outlawed, as compared to their desires with regard to other issues?
In general, in politics it’s quite common that the majority of the populace has a moderate preference to do X, a much smaller fraction of the populace has a very strong preference to not do X, and the desires of the minority win out. For the majority, the issue might not be important enough that they’d change their vote because of it, especially if the politician in question supports other issues who the people feel are more important. For the minority, however, the issue may be important enough to be the deciding factor in whom they vote. So the politican will maximize their votes by doing what the minority wants with regard to issue X, and what the majority wants with regard to everything else. At the same time, if the minority and majority vote for different politicians, then it’s beneficial for the elected politicians to barter votes, so that the majority “buys” the minority’s support for laws that they might not be able to pass otherwise, in exchange for giving the minority what they want on an issue that feels less important for the majority.
Of course, all of this presumes that the voters act “rationally”, and give their support to the candidates who most accurately match the desires of the voters. Pretty much everything that we know about voter behavior says that this isn’t the case. (Rationally was in scare quotes because, given how little influence a single vote actually has on an election, not bothering to figure out what your candidate actually does may in fact be the most rational use of your time.)
On a somewhat related note, non-slaveholders often bought goods that involved slave labor at some point in the process of their production. It’s quite possible that they at least thought that freeing the slaves would raise the cost of their tobacco, clothes ect. edited to add This is a bit more speculative, but I suspect that labor was significaly non-fungibal, and in particular that Northerner’s didn’t consider Southern agricultural jobs to be closely tied to their own labor market.
Also, poor white Southerners supported slavery because they had racial pride.