Last September, the Williams College psychologist Susan Engel had an opinion piece in the New York Times on the value of standardized testing as a means of assessing the quality of a child’s education.  Engel argued that there was scant evidence that these tests were of any value at all, and that they should be replaced by the many “promising techniques” that psychologists had already identified as valuable in assessing the learning of our children.

So what does this have to do with nutrition and weight control? Well, among the promising techniques, wrote Engel, was this one:

Researchers have also found that the way a student critiques a simple science experiment shows whether he understands the idea of controlling variables, a key component in all science work. To assess children’s scientific skills, an experiment could be described to them, in writing, and then they would explain how they would improve upon it.

So the value of controlling variables in a scientific experiment is something that a reasonably well-educated child supposedly understands. And what I want to know is why don’ t nutritionists understand it and those researchers out there doing diet trials and studying obesity and weight regulation. Because their failure to do so — and I would argue that it may be a willful failure — has led to what may be another of the great misconceptions in modern nutrition research. In particular, that carbohydrated-restricted diets are “valuable tools” in the arsenal against overweight and obesity, but they’re just one of the dietary tools.

This belief stems from the last decade of diet trials comparing carbohydrate-restricted diets (usually Atkins) to low-calorie, low-fat diets. Instead of thinking of low-carbohydrate diets like Atkins as deadly, which was formerly the case, nutritionists and dietitians (or at least most of them) now think of these diets as useful, just as other diets, low in calories or fats, are also useful. The idea now is that some people do well on carbohydrate-restricted diets and some people do well on low-fat diets, and maybe this is a result of whether they happen to be insulin sensitive or insulin resistant or maybe its just a product of their particular food tastes and preferences.

And this belief, of course, is based on the notion that we get fat for reasons other than the nutrient composition of the diet – probably because of some combination of our genes, our tendency to eat to much and our sedentary behavior – and so the diet that works best is the one that allows us to most comfortably restrict our intake of total calories.

This was the conclusion, for instance, of a 2008 article by Chris Gardner and his colleagues at Stanford, reporting on a subgroup analysis of their famous A to Z study.  (The trial is famous, at least, in the low-carb world, because the Atkins diet resulted in twice the weight loss of any of the three other diets tested, and it also did a better job of improving heart disease risk factors). In this follow-up study, Gardner and his colleagues reported that in each diet group — from the Atkins diet on the high end of the dietary fat to carbohydrate ratio to the Ornish diet on the low end — the subjects who actually adhered to the diet lost the most weight. Hence, their conclusion: maybe adherence to a diet is more important than the actual nutrient composition of the diet. Here’s the concluding paragraph:

The main findings of this weight loss study, presented in a previous report, indicated that while all three diet groups lost modest amounts of weight, the Atkins group at 12 months lost approximately twice the weight of the other groups. The findings presented here indicate that weight loss in the lowest tertile [third] of adherence was negligible in all three diet groups, and more pronounced in the highest tertile of adherence for each diet group. It appears that substantial differences in proportions of dietary macronutrients play only a modest role in weight loss success, and that success is possible on any of these diets provided there is adequate adherence. Getting individuals to adhere to whatever diet they choose to follow deserves more emphasis. It remains to be determined to what extent there is a need for dietary weight loss programs that are easier to adhere to vs identifying and addressing individual barriers to adherence, or both.

So the nutrient composition of the diet is less important than whether or not the subject can live with the diet and is willing to do so for as long as it takes — ideally, a life time.

This concept of low-carb diets being good for some people and low-fat for others  is invariably reinforced by the fact that most of us  know someone who has lost weight and kept it off on Weight Watchers or after reading Skinny Bitch or some other popular low-calorie diet book. As a result, we assume that dieting isn’t a one-sized fits all endeavor and that everyone is different – perhaps metabolically and hormonally, as well – and that what works for me won’t necessarily work for you, and vice verse.

So what does this have to do with controlling variables or even understanding the concept of controlling variables?

What researchers like Gardner and his colleagues do in these diet trials  (and it’s the same thing most of us do when we think about those people who succeed on conventional  diets or after reading diet books like Skinny Bitch) is make the assumption that a diet that is described as a “low-fat diet” is low in fat only and that’s why it works. And they also make the assumption that a diet that restricts total calories works (if it does) because it restricts total calories. Another way of saying this is that we all tend to assume — researchers and lay people alike — that when someone embarks on a low-fat diet, the only meaningful variable that changes in their diet is the fat-to-carbohydrate ratio. The ratio gets smaller. Fat consumption goes down and carbohydrate consumption goes up. And, by the same token, when someone tries to simply eat less, the only meaningful variable that’s changing is the total number of calories they’re consuming.

The most extreme or perhaps egregious example of this thinking was the recent publication by Gary Foster and his colleagues, comparing low-fat diets, as they described them, to low-carbohydrate diets. The title was “Weight and Metabolic Outcomes After 2 years on a Low-Carbohydrate Versus a Low-Fat Diet.” And here was the conclusion as stated in the abstract:

Successful weight loss can be achieved with either a low-fat or low-carbohydrate diet when coupled with behavioral treatment. A low-carbohydrate diet is associated with favorable changes in cardiovascular disease risk factors at 2 years.

So the way the media and the nutrition community treated this was as further evidence that nutrient composition of the diet makes little difference in weight loss — maybe low-carb works for some of us, but low-fat works for others — although,  in this case, maybe low-carb had some modest advantage when it came to heart disease risk factors.

But if you read this article carefully, you’d have noticed that there was another significance difference between the “low-fat” and low-carbohydrate diets. The low fat diet was a low-calorie diet also — “A low-fat diet consisted of limited energy intake (1200 to 1800kcal/d; less than or equal to 30 % calories from fat),” the authors explained. The low-carbohydrate diet was not calorie-restricted. And if Foster and his colleagues were being either intellectually honest or good scientists, they’d have defined the two diets to make this clear. Not  “low-fat” vs.  “low-carbohydrate”, but “low-fat, calorie-restricted” vs, “low-carbohydrate, calorie-unrestricted.”In other words they’d have acknowledged that there was at least one other variable that was different between the two experiments and had to be taken into account when interpreting the results — the amount of calories the subjects were instructed to consume. As we’ll see, there were also other variables that were changing, but this one — how much food can be consumed if desired — is a whopper.

It’s a whopper because it begs this question: is it the total calories consumed that is the variable determining weight loss? And, by the same token, is it the calories consumed (or expended) that determines how much weight we gain?

In this case, both diets resulted in roughly equal weight loss but those subjects randomized to the “low-fat” diet were instructed and counseled to semi-starve themselves (eat a maximum of 1500 calories for women, 1800 for men), while those counseled to eat low-carb were counseled and instructed not to worry about how much they ate and, one hopes, as this was an Atkins diet being prescribed, eat until they were full. So if weight loss is the same in both groups, doesn’t this suggest, at least, that weight loss can be independent of whether dieters semi-starve themselves or eat to satiety? And, if so, of course, wouldn’t you rather get to eat to satiety?

Had Foster and his colleagues understood what school children are supposed to understand, according to Engels,  “the idea of controlling variables, a key component in all science work,” they may have decided to control for calories and instructed both groups that they could eat as much as they want, rather than just the low-carbohydrate group. Or, had they had the money to spend, they might have cooked meals for both groups of subjects, say, 2700 calories a day – either low-fat or low-carb – and encouraged both groups to eat all the food prepared. Such an experiment would have gone a long way to “controlling” for calories consumed or for whether the subjects were allowed to eat to satiety or not. In doing so, it might have revealed something meaningful about whether the nutrient composition of the diet plays a role in weight loss or weight gain independent of calories, which is one of the critical questions here. I’d hazard a guess that it surely does, but I could be wrong. It would be an interesting experiment to do and I’ll write  considerably more on that in a later post.

As for the other mistake Foster, Gardner and their colleagues make when they assume that a low-fat, calorie-restricted diet (defining it correctly) is restricted only in fat, it’s the same mistake we make when we assume that someone who lost weight following Weight Watchers or after reading Skinny Bitch did it merely because something about these regimens got them to eat fewer calories and maybe fewer fat calories in particular. And this is the other mistake that suggests a lack of understanding of the idea of controlling variables.

Virtually any diet that significantly restricts the number of calories consumed, even a diet that is described as low-fat (because the subjects are instructed to reduce the proportion of fat calories they consume), will cut the total amount of carbohydrate calories consumed as well. This is just simple arithmetic. If we cut all the calories we consume by half, for instance, then we’re cutting the carbohydrates by half, too. And because these typically constitute the largest proportion of calories in our diet to begin with, these will see the greatest absolute reduction. If we preferentially try to cut fat calories, we’ll find it exceedingly difficult to cut more than 400 or 500 calories a day by reducing fat — depending on how much fat we were eating to begin with — and so we’ll have to eat fewer carbohydrates as well.

Put simply, low-fat diets that also cut significant calories will cut carbohydrates significantly as well, and often by more than they cut fat.

Here’s the math: Imagine we want to cut our daily calories from 2,500 to 1,500, hoping to lose two pounds of fat a week. And imagine that the nutrient content of our pre-diet meals is what the authorities consider ideal — 20 percent protein, 30 percent fat and 50 percent carbohydrates. That’s 500 calories of protein, 750 calories of fat and 1,250 of carbohydrates.

If we keep the same balance of nutrients but eat only 1,500 calories a day, we’ll be eating 300 calories of protein, 450 calories of fat and 750 calories of carbohydrates. We’ll be cutting protein calories by 200, fat calories by 300 and carbohydrate calories by 500.

Now let’s make this a “low-fat” diet and try to reduce our fat consumption from 30 percent of calories to, say, 25 percent of calories, which is significantly less than most of us will tolerate. We’ll now be eating 300 calories of protein, 375 calories of fat and 825 of carbohydrates. We’ll be cutting our fat calories by 375 a day, but we’re still cutting carbohydrates by 425. So even though the percentage of carbohydrates consumed on this “low-fat” diet goes up — from 50 to 55 percent — the absolute amount of carbohydrates consumed goes down, and goes down more so than does the calories from fat. And if we increase the amount of protein we eat, we’ll have to eat still fewer carbohydrates to compensate.

If we start off eating enough fat, as I said — say, 40 percent of our calories — we can actually cut fat calories more so than carbs, but carbs are still cut significantly. Imagine our 2500 calorie per day diet is 40 percent fat, 40 percent carbs and 20 percent protein. That’s 1000 calories of fat and carbs each, and 500 calories of protein. If we now cut that to a 1500 calorie diet that’s 30 percent fat and 50 percent carbohydrates, we’ll be eating 450 calories of fat, 750 calories of carbohydrates and 300 calories of protein. So fat calories will have dropped by 550 calories, but we’ll still have reduced carbohydrate calories by 250. Not an enormous amount but an amount that might still have an effect on the regulation of our fat tissue and so fat loss.

Here’s an example of how this plays out in a real dietary trial. Consider  an Israeli trial published in the New England Journal of Medicine in 2008 by Iris Shai and her colleagues.  This trial compared a low-fat, calorie-restricted diet to a Mediteranean, calorie-restricted diet to a low-carbohydrate Atkins diet, unrestricted in calories. And, you’ll notice here, too, having explained that the first two diets are calorie-restricted and the latter diet isn’t, Shai and company get lazy and shorten their labeling of the diets so that they leave out the critical variable of whether the dieters are instructed or not to semi-starve themselves.

In this study, Shai and her colleagues made an attempt to assess what their subjects were eating before the trial started, and then after 6, 12, and 24 months. Keeping in mind that the dietary records from these studies have to be taken with a grain of salt, here’s the relevant data:

Let’s concentrate on the low-fat, calorie-restricted diet and the low-carb, Atkins diet. The changes in dietary intake and nutrients for the “low-fat diet” are shown in the first column. As you can see after 24 months, the subjects eating the low-fat diet were supposedly restricting calories consumed on average by 572 calories. The reduction in carbohydrates consumed, though, was 330 calories (82.8 grams per day times 4 calories per gram), compared to only a 170-calorie (18.9 grams per day times 9 calories per gram) reduction from baseline in fat. So the “low-fat diet” reduced carbohydrates nearly twice as much as it reduced fat.

The low-carbohydrate diet, on the other hand (the third column), reduced carbohydrate calories by 520 calories per day (129.8 grams per day times 4 calories per gram) and fat calories by a mere 15 calories (1.7 grams/day times 9 calories per gram). So certainly the low-carb diet was correctly described as a low-carb diet, and the question we have to ask is maybe the weight loss seen in the low-fat diet was also due to the restriction in carbohydrates. It is quite possible that even low-fat, calorie-restricted diets work because they restrict carbohydrates and maybe the reason they don’t work as well as the low-carb diets is they don’t restrict them as much. Or maybe they don’t work as well, on average, because they also restrict fat calories when dietary fat has little or no effect on body fat accumulation. We don’t know if this is true or not, but it could be true, and until these researchers realize that another variable is changing significantly on these low-fat, calorie-restricted diets –  the amount of carbohydrates consumed — they’ll never bother to test it or take it into account in their interpretation of these clinical trials, and we’ll never know.

Now, here’s yet another variable that’s changing on these diets, and this one the researchers ignore entirely and make no attempts to quantify — the quality of carbohydrates consumed. Any subject in these diet trials and anyone who tries a serious weight loss program on their own (the twinkie diet, perhaps, not included) will make a few consistent changes to what they eat. And they’ll do this regardless of the instructions that they’re given or the diet to which they’re randomized in the trial.

Specifically, they’ll get rid of or cut way back on the high-glycemic index carbohydrates and the foods or drinks with the high sugar or HFCS content. They’ll do so  because these foods are the easiest to eliminate and the most obviously inappropriate for anyone trying to get in shape. (And because for a almost 200 years these foods have been considered uniquely fattening.) They’ll stop drinking beer, for instance, or at least drink less beer or drink light beer instead. They might think of this as cutting calories, but the calories they’ll be cutting will be carbohydrates and, more importantly, they’re liquid, refined carbohydrates that are exceedingly easy to digest and so, perhaps, exceedingly fattening.

They’ll stop drinking caloric sodas – Coca Cola, Pepsi, Dr. Pepper – and replace them either with water or diet sodas. In doing so, they’ll  be removing not just  liquid carbohydrates but specifically sugars — sucrose or HFCS. The same is true of fruit juices. An easy change in any diet is to replace fruit juices with water. Dieters will get rid of candy bars, desserts, donuts and cinnamon buns. Again, they may perceive this as calorie-cutting – and maybe even a way to cut fat, which it is – but they’ll also be cutting carbohydrates, and specifically sugars with their high fructose content. And if sugars with their high fructose content are uniquely fattening as significant evidence suggests, then this reduction in sugar content may be precisely why the diets work.  Starches like potatoes and rice, refined carbohydrates like bread and pasta, may also be replaced in these diets — even “low-fat” diets — by green vegetables and salads or at least whole grains, because for the past 30 years, we’ve been all told to eat more fiber and to eat foods that are less energy dense and less processed.

Even the very-low-fat diet made famous by Dean Ornish restricts all refined carbohydrates, including sugars, white rice and white flour. This alone could explain any benefits that result. Ornish’s rationale, as he described it in 1996 is a familiar one: “Simple carbohydrates are absorbed quickly and cause a rapid rise in serum glucose, thereby provoking an insulin response. Insulin also accelerates conversion of calories into triglycerides, [and] stimulates… cholesterol synthesis.”

Simply put, anyone who tries to diet by any of the more accepted methods (i.e., Weight Watchers), and anyone who decides to “eat healthy” as its currently defined, will remove the carbohydrates from the diet that may be — if the carbohydrate/insulin hypothesis is correct — the most fattening. And if they’re trying to cut calories, they’ll be removing some number of total carbohydrates as well. And if these people lose fat on these diets, this is a very likely reason why.

The same is likely to be true for those who swear they lost their excess pounds and kept them off by taking up regular exercise. Rare is the individual who begins  running or swimming or doing aerobics regularly with the goal of losing weight and then doesn’t make any concomitant changes in what he or she eats. Rather beer and soda consumption will be reduced; sweet consumption will be reduced, and easily digested starches and high-glycemic index carbs are likely to be replaced by green vegetables and carbohydrates with a lower glycemic index.

So here’s the lesson, the moral of this story: before we assume that low-carbohydrate diets are just one tool in the dietary arsenal against overweight and obesity, and before we assume that everyone is different and that some of us lose weight and keep it off because we eat less fat (and more carbohydrates) and some because we cut carbs (and so eat maybe more fat),  we should make an effort to understand the concept of controlling variables and look to see which variables are really changing and by how much. Because it’s quite possible that the only meaningful way to lose fat is to change the regulation of the fat tissue, and the science of fat metabolism strongly implies that the best way to do that, if not the only meaningful way, is by reducing the amount of carbohydrates consumed and/or improving the quality of those carbs we do consume.

Now, one note about comments that I should have made in my last (and first) blog. I appreciate everyone who comments, but time constraints (earning a living, participating in my family life, etc.) makes it necessary that I keep my responses to a minimum. So I am going to thank everyone in advance for their comments. I will be reading all of them (up to the point, at least, that they degenerate into arguments between two or three particularly vociferous and contentious individuals), but I will be responding only to those that raise particularly interesting questions or issues, or point out any bone-head mistakes I may have made that need to be fixed.