Madagascar is out of the question. It has been inhabited since 200 BC, when Indonesians (Yes, I'm not joking, Indonesians) first reached the island and chose to live there rather than return to their distant homelands. If this mythical race Insider's constructing is supposed to have lived far from any other human settlements, the heavily populated island and the smaller islands nearby, will not do. Not only because of the "indigenous" population, but also because the islands were frequently visited by Arab and East African traders and explorers. No unknown race could have ever lived there without getting found and exterminated. In addition to all this, nobody has ever found ruins or traces of any ancient civilisations there, making it even more ridiculous to locate the story to Madagascar.
Instead, I suggest two other possible locations for this purpose: Saint Helena or any of the Canary Islands.
The former was allegedly uninhabited until the Portugese found it in the 16th century, but as it is of volcanic origin, it isn't really far-fetched to suggest that a part of it may have collapsed back into the ocean a couple of centuries earlier, destroying all evidence of former inhabitants and their culture. The island itself is barren, has almost zero endemic animal species and no fruit bearing trees, making fish and seabirds the only source of nutrition for anyone living there. Saint Helena is also a very remote island, as the wiki-article also points out. Whether it is a pro or a con, I don't know, but this means that almost no other human ever set foot on the island. Not half bad, I say.
The latter location is a tricky one. The Canaries are so close to both Europe and North Africa that they were inhabited centuries earlier; the first recorded evidence of human settlements dates back to 1000 BC. All of the major Mediterranean powers have visited the Canaries at some point, some slaving and killing the aboriginals, some just stopping by. Not a very good place you might think. However, there is something that piques interest about the history of these islands: claims of inhabitants from times even further back in the history of mankind. Supposedly a mythical civilisation that could work stone and build sophisticated buildings using it lived on some of the smaller islands before the aborigines first arrived. Why didn't they move there earlier? Perhaps they were afraid of something..
Of course, there are some less well-known smaller islands that could be used as well, but I didn't think listing them would do any good at this point. Not when these few bigger islands are equally good or even better than the others. Note though, that these were mere suggestions. If you find something more fitting yourself, feel free to use it. I'm a huge fan of realism in fantasy myself, so I won't advise coming up with an island that has never existed in reality, but that's one possibility too.
Hope I've helped even the slightest bit.