Thank you, Mer, and everyone for posting this thorough summary of our efforts.
For community transparency, I will share here our review of the review notes and feedback, including points highlighted above:
Hello Stef and Mer,
Thank you again for allowing our review team a little extra time to understand the nuances of what we were working on and the work that has been refined by the working group to create the latest version. We have submitted our notes in the shared team document.
We wanted to provide some feedback about the review process as it exists now. We’re scoping this specifically to Abdoulaye’s and my review of OLMo from AI2. We know we have other threads to catch up on with you, but wanted to give you notes about our experience sooner than later.
Things that worked well for us
- This model was a recent launch - blogs consolidating information, and their source repositories, were all mostly up-to-date and easier to link across sources.
- It was VERY helpful to have a shared spreadsheet with other reviewers’ notes and guidance. This helped us understand where people had come from before in their work, and where others might have similar questions.
- We really appreciate the email thread between Mer and the other reviewers. This allowed us to ask questions we weren’t sure about, and see what general guidance was being given to the group. This also kept the conversation “in the room” to the people who had volunteered, as we sorted out our understanding of the work, and didn’t open up these questions to scrutiny from non-volunteers.
Things that were challenges
- The current component/system framework is not easily contained or bounded by current platforms or technology frameworks. This presents a challenge for reviewers AND developers to track down all the parts AND licenses surrounding a system.
- tl;dr > There is no “one and done” place to see artifacts, licenses, and terms & conditions attached to each component, or to understand what could apply to the system as a whole.
- The current component framework, even with the reference paper, was not clear for us which type of artifact should fall under each component. For example, in our notes, you’ll see questions like “Is this talking about the publication, the white paper, the source code, ….”
- tl;dr > This potential compounding of different kinds of artifacts together made it challenging to track down the legal document for a specific component, when we couldn’t determine which one applied or if there was any at all (in some cases - such as where white papers were listed without a copyright in the directory OR the paper.)
Where we felt blocked
- We did not feel equipped to make a decision about whether a copyright license attached to anything other than software (and even these!) were “compliant” or “conformant” with the OSD.
- tl;dr > We would appreciate guidance from the OSI’s licensing committee, maybe in partnership with groups like Creative Commons, to develop an initial list for reviewers for artifacts that are not software.
cheers,
amanda casari
Google Open Source OSS+AI Lead