Should testing data be made available to *use* an AI system?

(Reopening this topic, as it is becoming increasing relevant to current discussions Draft v.0.0.9 of the Open Source AI Definition is available for comments - #15 by Shamar )

I beg to disagree.

If you want to validate the claims of a particular Ai system (as you would if you have the freedom to study it), you will absolutely need all the original data it was used to test it.

It does not matter that you can create a new test dataset which could invalidate the original claims, you need to test the original conclusions against the original (train and test) data.

As with any other Open Source product, most people are only users; some will compile it; a few will study it and develop new features or correct bugs.

You need to have access to all parts to be able to exercise your four freedoms.

2 Likes