This should work as intended and filtering on false, should also return the posts that have no set values.
You could check if you’re sorting on that column as well, since sorting removes not set values by default. If you see the ‘Reset Sorting’ button on top of the table, you could try to click that and see if it changes anything. Or you could try to sort on another column. It is also possible to change this default behavior for sorting on our settings page on the settings tab.