At the outset of this section, Parsons writes that one point at which "reservations about standard first-order logic as the universal measure of ontology can affect the notion of mathematical object is the ancient question whether reference to objects is necessarily reference to objects that exist."
A comment before proceeding. Note that Parsons had earlier (Secs 1 to 4) proposed that (1) "speaking of objects just is using the linguistic devices of singular terms, predication, identity and quantification" to make serious, and indeed true, statements. And defending that view about, so to speak, the measure of what objects we are committed to falls short of saying that (2) standard first-order logic is the universal measure of ontology in general. Resisting the more sweeping claim is quite consistent with accepting Parsons's initial Fregean claim about objects. Not that I'm suggesting that Parsons thinks otherwise. I'm just emphasizing that if (e.g. as a Fregean) you are not persuaded by Parsons Sec. 5 suggestions, and hold that we are committed to entities that are not objects, then you can accept formulation (1) without accepting (2).
Anyway, what of reference to objects that is not reference to objects that exist? Parsons discusses Meinongian views in some detail (this is one of the longest sections in the book). Here's part of his final summary of the discussion.
We are left with the question whether the "true" meaning of the existential quantifier is [i] the permissive Meinongian one [allowing quantification over objects that do not exist], [ii] existence that allows freely for abstract objects but that rules out impossibilia, or [iii] something like actuality. The logic based concept of object does not decide between these alternatives, although, once it has been set forth, the case for [iii] is weakened. But in order to understand the notions of object and existence in mathematics we have to put more flesh on the bare form given by formal logic. We need to fill out the logic-based conception by looking at cases. ... [C]onsiderations proper to mathematics will not lead us to favour [i] over [ii]. General as the notion of object in mathematics is, there is still a constraint of possibility, coherence, or consistency that objects postulated in Meinongian theories are allowed to violate.
The talk here of having to "fill out the logic-based conception" might initially seems surprising given what has gone before. But, though he is not entirely clear, I assume that what Parsons means is simply this: the Fregean thesis is that objects are just whatever are we have to construe terms that behave in the right sorts of way in true sentences as referring to. So, to fill out that general template view about objects, we have to say what kinds of sentences we do in fact accept as being true. If we e.g. take statements like "Sherlock Holmes is more famous than any living detective" and "There's a fictional detective who is more famous than any living detective" at face value as true claims then (the suggestion goes) we have to accept (i) the Meinongian line that there are objects that do not exist. If we paraphrase away apparent talk of fictional objects and the like, but accept that there are true mathematical statements talking of numbers, sets, etc., then (ii) we are not committed to non-existent objects, but have to accept that there are abstract objects which aren't "actual". If we insist on also paraphrasing away apparent straight talk of numbers (e.g. construing it as governed by an operator "in the arithmetical fiction ..."), then perhaps (iii) we may only be committed to actual objects.
Parsons is sceptical about whether we have any need "to admit into the range of our quantifiers such objects as the golden mountain, the round square, Pegasus and Sherlock Holmes", though it is not his concern to argue for this here. But he does argue that "considerations proper to mathematics" don't give any impetus for preferring the Meinongian views (i) over (ii). Mathematics doesn't countenance impossibilia like the round square, or present itself as fictional discourse. As to (iii), I assume Parsons thought is that a critic of our common-or-garden standards of mathematical truth on the basis of a metaphysical repudiation of abstract objects is (in danger of) getting things upside down, at least by the lights of the truth-first, "logic-based conception" of objects, according to which we don't have a handle on the notion of an object except via a prior grip on the notion of truth for the relevant object-referring statements.
If this reading of Parsons is right, then I agree with him.