Maybe your boss wants the place to herself in the morning and she’s actually annoyed that you’re getting there early, too.” Vanderkam also thinks a lot of people aren’t willing to consciously build breaks into their daily schedule out of fear they won’t be seen as hard workers.