I think the transfer_efficiency example is kinda dishonest (also kinda wrong: the right-hand column uses an upper figure of 0.95 but the left-hand column uses 0.9). Squiggle apparently has special-case support for the special case of a lognormal variable where you know the lower and upper fifth-percentile values. That’s nice, I guess, but it really is quite a special case.
Unless I am going to be spending all my putative time in Squiggle specifying fifth-percentile values of lognormal variables, that example is of no use to me without knowing what happens if I want to specify the tenth-percentile values instead, what happens if I want a normal distribution, etc.
I had a look in the Squiggle documentation, and it looks to me as if the “to” syntax is only for 5th-to-95th percentiles, and only for normal distributions where those quantities have different signs and lognormal distributions where they have the same sign. (Which means, e.g., that if you are calculating those quantities then you cannot safely use “to” unless you know for sure what the signs will end up being.) If you have Squiggle code that uses “to” and then you realise that actually you wanted to use the 10th and 90th percentiles, or a different distribution, or a normal distribution even though both those quantiles are positive, you need to throw away that one line and write something completely different that does the same untidy thing as those “15 lines” of Python code. Is the corresponding Squiggle code in that case actually any shorter or clearer?
Also, those “15 lines” of Python code corresponding to the “1 line” of Squiggle code include
three lines of imports that will happen once at the start of whatever thing you’re doing and don’t need repeating for each uncertain quantity you work with
one line initializing a variable that is never actually used in the code
one line double-checking that the earlier lines produced something like the right result
and if you were in fact writing something in Python that makes substantial use of distributions specified by giving a couple of quantiles, you would write a function containing the boilerplate once and never have to worry about the fiddly details again.
(On the other hand, if you’re only doing it once then indeed the boilerplate is a substantial fraction of your code. But if you’re only doing it once then I bet the broader advantages of a general-purpose programming language will outweigh the local advantages of a domain-specific language specializing in doing neatly something you only need to do once.)
For the avoidance of doubt, this is not intended to imply that Squiggle is bad. Just that that example seems super-unfair.
[EDITED to add:] I’m also really confused, because the actual Squiggle notebook linked to from the discussion this example supposedly comes from (1) doesn’t in fact define transfer_efficiency this way, (2) does consider at least one lognormally distributed quantity determined from estimates of its quantiles, namely consumption_prior, and (3) handles that quantity … by giving explicit numerical values for the mean and stddev of the underlying normal distribution. There is a quantity in the spreadsheet called transfer_efficiency, it is beta-distributed not lognormal, and its distribution is also specified by giving magic numbers for the distribution parameters rather than saying where two of its quantiles are.
I’d be really interested in a head-to-head comparison with R on a bunch of real-world examples of writing down beliefs that were not selected to favor either R or Squiggle. R because at least in part specifying and manipulating distributions seems to require less boilerplate than in Python.
0.75 to 0.95 vs.0.75 to 0.9 is strictly my transcription bug, not being careful enough.
In general I wasn’t auditing the code from the Jonas Moss comment, I just stepped through looking at the functionality. I should’ve been more careful, if I was going to make a claim about the conversion factor.
You’re kinda right about the question “if it’s a constant number of lines written exactly once, does it really count as boilerplate?” I can see how it feels a little dishonest of me to imply that the ratio is really 15:1. The example I was thinking of was the Biological Anchors Report (“Ajeya’s Timeilnes”), those notebooks have lots of LOC in hidden cells, but the relative cost of those goes down as the length of the report goes up. All that considered, I could be updated to the idea that the boilerplate point is moot for power users (who are probably able and willing to provide that boilerplate once per file), but I would still be excited about what is opened up for more casual users.
You’re right that, or your comment is suggesting to me indirectly that, squiggle, having not yet provided a way to give non-default quantiles with the to syntax, hasn’t done anything to show that it’d really beat hand-crafted python functions, to accomplish this.
Re the underlying squiggle notebook concerning GiveDirectly and so on, I’ve flagged your comment to Sam (it’s something else I haven’t taken a close look at).
Thanks for the flag! I might not be understanding correctly, but I don’t think there’s a problem here with the actual underlying code just my explanation of it (we all hate magic numbers). Which is very fair enough, the notebook is much too dense for my liking. It’s still a work in progress!
I agree! The Squiggle team is looking to create different quantiles for different distributions. I’ve needed them on several occasions. You can check out the discussion on GitHub here. It’s on my todo list.
I think the
transfer_efficiencyexample is kinda dishonest (also kinda wrong: the right-hand column uses an upper figure of 0.95 but the left-hand column uses 0.9). Squiggle apparently has special-case support for the special case of a lognormal variable where you know the lower and upper fifth-percentile values. That’s nice, I guess, but it really is quite a special case.Unless I am going to be spending all my putative time in Squiggle specifying fifth-percentile values of lognormal variables, that example is of no use to me without knowing what happens if I want to specify the tenth-percentile values instead, what happens if I want a normal distribution, etc.
I had a look in the Squiggle documentation, and it looks to me as if the “to” syntax is only for 5th-to-95th percentiles, and only for normal distributions where those quantities have different signs and lognormal distributions where they have the same sign. (Which means, e.g., that if you are calculating those quantities then you cannot safely use “to” unless you know for sure what the signs will end up being.) If you have Squiggle code that uses “to” and then you realise that actually you wanted to use the 10th and 90th percentiles, or a different distribution, or a normal distribution even though both those quantiles are positive, you need to throw away that one line and write something completely different that does the same untidy thing as those “15 lines” of Python code. Is the corresponding Squiggle code in that case actually any shorter or clearer?
Also, those “15 lines” of Python code corresponding to the “1 line” of Squiggle code include
three lines of imports that will happen once at the start of whatever thing you’re doing and don’t need repeating for each uncertain quantity you work with
one line initializing a variable that is never actually used in the code
one line double-checking that the earlier lines produced something like the right result
and if you were in fact writing something in Python that makes substantial use of distributions specified by giving a couple of quantiles, you would write a function containing the boilerplate once and never have to worry about the fiddly details again.
(On the other hand, if you’re only doing it once then indeed the boilerplate is a substantial fraction of your code. But if you’re only doing it once then I bet the broader advantages of a general-purpose programming language will outweigh the local advantages of a domain-specific language specializing in doing neatly something you only need to do once.)
For the avoidance of doubt, this is not intended to imply that Squiggle is bad. Just that that example seems super-unfair.
[EDITED to add:] I’m also really confused, because the actual Squiggle notebook linked to from the discussion this example supposedly comes from (1) doesn’t in fact define
transfer_efficiencythis way, (2) does consider at least one lognormally distributed quantity determined from estimates of its quantiles, namelyconsumption_prior, and (3) handles that quantity … by giving explicit numerical values for the mean and stddev of the underlying normal distribution. There is a quantity in the spreadsheet calledtransfer_efficiency, it is beta-distributed not lognormal, and its distribution is also specified by giving magic numbers for the distribution parameters rather than saying where two of its quantiles are.I’d be really interested in a head-to-head comparison with R on a bunch of real-world examples of writing down beliefs that were not selected to favor either R or Squiggle. R because at least in part specifying and manipulating distributions seems to require less boilerplate than in Python.
Thanks!
0.75 to 0.95vs.0.75 to 0.9is strictly my transcription bug, not being careful enough.In general I wasn’t auditing the code from the Jonas Moss comment, I just stepped through looking at the functionality. I should’ve been more careful, if I was going to make a claim about the conversion factor.
You’re kinda right about the question “if it’s a constant number of lines written exactly once, does it really count as boilerplate?” I can see how it feels a little dishonest of me to imply that the ratio is really 15:1. The example I was thinking of was the Biological Anchors Report (“Ajeya’s Timeilnes”), those notebooks have lots of LOC in hidden cells, but the relative cost of those goes down as the length of the report goes up. All that considered, I could be updated to the idea that the boilerplate point is moot for power users (who are probably able and willing to provide that boilerplate once per file), but I would still be excited about what is opened up for more casual users.
You’re right that, or your comment is suggesting to me indirectly that, squiggle, having not yet provided a way to give non-default quantiles with the
tosyntax, hasn’t done anything to show that it’d really beat hand-crafted python functions, to accomplish this.Re the underlying squiggle notebook concerning GiveDirectly and so on, I’ve flagged your comment to Sam (it’s something else I haven’t taken a close look at).
Thanks for the flag!
I might not be understanding correctly, but I don’t think there’s a problem here with the actual underlying code just my explanation of it (we all hate magic numbers). Which is very fair enough, the notebook is much too dense for my liking. It’s still a work in progress!
I agree! The Squiggle team is looking to create different quantiles for different distributions. I’ve needed them on several occasions. You can check out the discussion on GitHub here. It’s on my todo list.
Just letting you know that you seem to have double-pasted the 3rd bullet point.
oof, good catch, fixed.