Not sure, but could, "not looking for work" also mean retired in many cases. Baby boomers are a large group and could be a big part of the number represented in the graph.
Right. Here's the breakdown of what BLS uses (not seasonally adjusted):
Adult Civilian Non-Institutional Population (everyone 16 and older, not in prison, mental institute, or the military):236,743,000
Labor Force: Employed (worked in the reference week) 139,132,000 + Unemployed (didn't work in the reference week but lookin the previous 4 weeks) 14,407,000 = 153,539,000
Not in the Labor Force (did not work or look for work): 83,204,000
Breaking down Not in the Labor Force further...of those 83,204,000, 5,618,000 say they want a job. Of those, 2,323,000 say they are available to work and have looked sometime in the previous 12 months. These are called the Marginally Attached. Of those, 861,000 have stopped looking because they don't think they'll find work. These are called "Discouraged Workers." The remaining 1,462,000 marginally attached didn't look for non job-market reasons...transportation, child care, disability, illness, etc.
Now, the Not in the Labor Force number has grown by percentage, not just pure numbers, but percentage wise as many people have stopped looking for work and fewer people have started. The Labor Force Participation Rate (65%) hasn't been this low since 1986. But keep in mind it went up steadily since the 70's and more women entered the labor force.
The Employment Population ratio, is miserable as well at 58.5% The worst since the early 80's. Very steep decline. Hopefully, we're near the bottom of that, but how near is questinable.