Probably not that unlucky actually. you had a 1/16 chance that one of the two random IVs would be perfect, but at the same time there isn't a 100% chance of finding one with 4 pre-rolled perfect IVs (tests have been done, and even after 50+ chains some of them have 3). So it's possible that some of your 4-IV Dittos were actually 3-IV dittos with a perfect roll on a 4th… and finding one with 4 is already somewhat lucky.
Anyway I don't see much point in the time investment on finding one with more than 4. Just use two dittos that when combined have all of the IVs you want. It will only add one fairly quick step to the breeding process. I only caught 2 dittos but I feel lucky because they both have 4 IVs that do cover all the stats between them, and one is Adamant while the other is Jolly.