Definition Validation: Seeking Volunteers

Open Source AI
1

CONTEXT Last week Stefano announced v. 0.0.8, the first feature complete version of the Open Source AI Definition (OSAID).

ASK We are now seeking volunteers to validate that definition by using it to review additional AI systems that are self-described as open. If you are interested, please comment below or DM me.

TIME We would like to complete this validation task by Monday, May 20th.

TASK You will have a spreadsheet (example below) in which you locate and link to the license, research paper, or other document that grants rights or provides information for each required component. You will then indicate in each cell whether the document Allows or Restricts the ability to study use, modify, or share that component.

Screenshot 2024-05-01 at 10.08.59 AM
Screenshot 2024-05-01 at 10.08.59 AM1920Ă—950 194 KB

Example: spreadsheet used to review Llama 2 according OSAID v. 0.0.6

SYSTEMS We are interested in reviewing about ten self-described open AI systems as part of this definitional process. Four (marked *) have already been reviewed by the workgroups. Below are some more systems we are interested in being part of this validation task. If there is another system you would like to review, please say so.

  1. Arctic JesĂşs M. Gonzalez-Barahona
  2. BLOOM * Danish Contractor, Jaan Li
  3. Falcon Casey Valk, Jean-Pierre Lorre
  4. Grok Victor Lu, Karsten Wade
  5. Llama 2 * Davide Testuggine, Jonathan Torres, Stefano Zacchiroli, Victor Lu
  6. Mistral Mark Collier, Jean-Pierre Lorre, Cailean Osborne
  7. OLMo Amanda Casari, Abdoulaye Diack
  8. OpenCV * Rasim Sen
  9. Phi-2 Seo-Young Isabelle Hwang
  10. Pythia * Seo-Young Isabelle Hwang, Stella Biderman, Hailey Schoelkopf, Aviya Skowron
  11. T5 Jaan Li

TO VOLUNTEER Comment below or DM me if you would like to volunteer. Anyone who can complete the task may do so, either solo or as a group. If you are a creator or advisor on the system you are reviewing, we will also need to identity an unaffiliated individual to review the system, so please disclose that when you volunteer. Your name and organizational affiliation will also be made public as part of our transparency policy.

Women, trans, and non-binary folx, black, indigenous, latine/o/a, and other people of color, immigrants, people with disabilities, and people from poor and working class backgrounds are encouraged to respond.

Thanks :slightly_smiling_face:

EDIT: I’ll update the list above with volunteer names so it’s clear where there is greatest need.
(last update: May 14th @ 11:03 am PDT)

3 Likes
2

I volunteer to help with a review!

I have a conflict of interest with #10, T5 (same employer), but could take the lead or assist on any others.

I would prefer to start w/ OLMo, if that isn’t taken yet.

3

Thank you, @amcasari! You’ve got review on OLMo. I’m going to make up the new review spreadsheet today and will email you with further details.

1 Like
4

Here is another one claiming to be “truly open”: Can we get someone to review it, please?

2 Likes
5

There are two kinds of systems I don’t see listed that I would like to see.

  1. Non-foundational models.
  2. Systems which are not based on deep learning (and, preferably, not even on machine learning).

That said, if I can only suggest specific systems, I have at least two:

1 Like
6

Good thinking, thanks! I think you can get started by cloning this table structure and fill in the details.

1 Like
7

Heya Mer,

I love this exercise! I’ve been doing this lightly across systems recently but not to this same depth and level of certainty. In fact, Snowflake Arctic is one I looked more closely at, so I’m happy to help with this one if needed.

Otherwise, I have no preferences and will help wherever needed. Would you like to choose several for me, prioritize them if possible? I’m not aware of my having any affiliation with these systems, and Open Community Architects (OCA) is a neutral consultancy in those regards (aside from a bias for Open.)

  • Karsten (quaid)
1 Like
8

Thanks, @quaid. I think Arctic is well-covered, but let me see if anyone else needs help. If not, you can just pick a system that works for you. I’ll get back to you soon.

9

@quaid I got no requests from the current volunteers, so I’m going to put you on Grok, which is a new system with only 1 volunteer. Please DM me your email address so I can add you to the reviewer chat and spreadsheet.

10

Two models I hadn’t thought about mentioning previously, but are rather interesting, are Segment Anything, by Meta and Whisper, by OpenAI.

What makes them interesting is the fact that they sparked a lot of attention when they were published and they are built by companies that usually make in-house models (OpenAI) or often release models under proprietary licenses (Meta), but in this case were released under open source licenses (both code and weigths): Apache 2.0 and MIT license respectively.