The Skill and The Wit (spoilers all RotE books)

[Farseer]

Yes, just found this from Chade, speaking to Fitz, in Chapter Eleven of FE, 'Chade's Tower':

...But others point back a time, to the Piebald Prince and several other oddlings in the Farseer line, to say, "No, the taint is there, down in the roots, and Prince Dutiful is a shoot from that line."

It's after this that they mention that "the Piebald Prince died without issue" (confirming what I said earlier) but of the 'other oddlings' there is no mention. This suggests that there were other Witted ones who popped up earlier in the Farseer line.

Not sure if this is the quote you were after, SwiftWitted? I know that Fitz talked about the Witted in other areas such as the little chapter intros...