Literally it means: "We were all for worse." Since it comes from Candide which is a French satire first published in 1759 by Voltaire, a philosopher... the novella has been widely translated, with English versions titled Candide: or, All for the Best; Candide: or The Optimist. It begins with a young man, called Candide, who is living a sheltered life in an Edenic paradise and being indoctrinated with optimism by his mentor, Pangloss. The work describes the abrupt cessation of this lifestyle, followed by Candide's slow, painful disillusionment as he witnesses and experiences great hardships in the world. Candide is characterised by its sarcastic tone, as well as by its erratic, fantastical and fast-moving plot. A picaresque novel with a story similar to that of a more serious bildungsroman, it parodies many adventure and romance clichés, the struggles of which are caricatured in a tone that is mordantly matter-of-fact. Voltaire ridicules religion, theologians, governments, armies, philosophies, and philosophers through allegory; most conspicuously, he assaults Leibniz and his optimism. Best regards

I understand it as "we were all agonizing, suffering a lot, close to death". I imagine it's a shortcut for "à l'article de la mort" or "avec l'impression d'être aux portes de la mort"…