I’ve Heard It Both Ways

Cleaning up Gmail with App Script

2021-02-22T00:00:00+00:00

After several years working in the open source community, I’ve ended up with a lot of emails from Gerrit code reviews, Google Group mailing lists, and various GitHub notifications. And when I say a lot, I mean my Google storage for Gmail, Photos, Drive, and others were getting closer and closer (78%) to the max free capacity.

To be fair, I do know quite a bit of that is from other things besides email. I have several large files in Drive, and a few longer videos from when my kids were younger sitting in Google Photos. But the weird thing that struck me was that even though I would delete emails from things like code reviews, they would still show up when I searched by inbox for various terms. I thought the way it was supposed to work was that I would delete an email, it would sit in my Trash for 30 days or so, then Gmail would clean things up for me. Based on what I was getting from my inbox searches, that apparently isn’t always the case.

So rather than allow things to keep up slowly creeping up to max out my storage quota, I decided to take a look at what I could do to clear out this old cruft. At first I would perform some searches, use the Select All to select all 50 messages on the page, and delete them. And the next page. And the next.

That was going to take a long time. It wouldn’t give me the message count from queries like from:review@openstack.org, but it looked like it was in the 1000’s. Even with no where to go, I didn’t have time to click through all that.

Automation (sorta) to the rescue

I’m probably late to the party - based on forum results, it appears several years late to the party - but it didn’t take me too long to stumble on the Google Apps Script site.

I started by looking to see if there was an API for interacting with Gmail. Turns out there is, for several different languages. But the Quickstart Guide included a link to how to get started with Apps Script. That has a nice walkthrough of getting a simple script set up to interact with Gmail.

From there, it was pretty easy to adapt the example they gave to work with messages instead. Looking through the API documentation, I found the message object has a moveToTrash() method. That led me to come up with this quick example:

function deleteMessages() {
  var count = 100
  while (count > 0) {
    var threads = GmailApp.search('from:review@openstack.org', 0, 10)
    count = threads.length
    for (var i = 0; i < count; i++) {
      var thread = threads[i];
      for (j = 0; j < thread.getMessageCount(); j++) {
        var message = threads.getMessages()[j];
        message.moveToTrash()
      }
    }
  }
}

It’s fairly easy to work with the Gmail API. To break it down bit, here are some key parts.

Getting threads

var threads = GmailApp.search('from:review@openstack.org', 0, 10)

This queries the Gmail API, using a query just like you would use within the Gmail UI itself. In fact, I highly recommend first running your query in the UI and verifying it returns what you expect it to, then copying and pasting that query string into the script. We are deleting messages here, so you probably should be very cautious about what gets automatically deleted.

The two numbers, 0, 10, are the starting count and the max number of threads to retrieve. Starting off, this query would return a very, very large amount of messages. Pulling that all across the network and into memory would not be very nice. So chunking this out into smaller amounts is a lot more efficient. I would rather have to run the script repeatedly than try to load my entire inbox into the script.

And speaking of running repeatedly, there apparently is a hard coded 5 minute timeout on script execution. So although this speeds things up quite a bit over manually deleting, it still does take time. So one way or another I was going to have to keep rerunning the script to get everything cleared out anyway.

Bringing out the trash

message.moveToTrash()

Most of the rest of the code is just looping through results. The next code of interest is the actual deletion. This is accomplished by calling the moveToTrash method on the message object. Just as it sounds like, it moves the message from wherever it is into your Trash folder.

Slight optimization

This was great progress. I ran it a few times, and I could see from my search result that things were getting cleared out. It was taking a while to execute, so I looked through the API docs some more. It turns out, I was wasting some time iterating through all those messages within the thread.

var threads = GmailApp.search('from:review@openstack.org', 0, 10)
count = threads.length
for (var i = 0; i < count; i++) {
    var thread = threads[i];
    for (j = 0; j < thread.getMessageCount(); j++) {
        var message = threads.getMessages()[j];

That’s looping through each thread that is returned from the query, and in turn, retrieving and looping through each message within the thread. For things like OpenStack code reviews, some patches with a lot of feedback, updates, and CI comments, that can mean quite a few messages per thread.

Luckily, the thread object itself also has a moveToTrash method. So knowing that, I was able to clean that up quite a bit:

var threads = GmailApp.search('from:review@openstack.org', 0, 10)
count = threads.length
for (var i = 0; i < count; i++) {
    thread.moveToTrash()

This was definitely more efficient.

Space, the final frontier

I still had a problem with the script though. Or maybe, more of a problem with my patience and my trust in the Trash actually being emptied after 30 days. What I really wanted was to immediately and completely delete these messages and get them out of there.

None of the examples I read in the official docs mentioned it, though that could very well just be my impatience and not wanting to read up on more than I really needed to. But I found an old forum posting somewhere that mentioned a remove call.

DANGER: Only do this if you are very careful!

Seriously, don’t blame me if you mess things up. I warned ya!

This call will actually do exactly what I wanted. It gets the message completely out of there right away. Of course, with scripting and automation, there is no “Are you really sure?” dialogs to click and slow you down. If you get the query wrong, too late, it’s gone.

So again being very cautious about the query string being used to retrieve the threads, I tried one more approach.

Stepping back

Now the only bad thing here (OK, probably not the only bad thing) is that the remove call only works on message IDs. Not threads.

I just had to roll back some of the optimization I had done before and go back to iterating through the messages in the thread again. It slowed things down, but ultimately it was still faster, at least to me, because it would permanently get rid of the messages. The final version I ended up with was then:

var threads = GmailApp.search('from:review@openstack.org', 0, 10)
count = threads.length
for (var i = 0; i < count; i++) {
    var thread = threads[i];
    for (j = 0; j < thread.getMessageCount(); j++) {
        var message = threads.getMessages()[j];
        Gmail.Users.Messages.remove('me', message.getId());

That last line takes a user ID, but luckily it has a special value of me that will recognize the current user. And the ID of the message to remove, retrieved by calling the getId() method on the message. After running this version over and over after the 5 minute timeouts, I started to see my storage usage number start to drop down.

More cleanup

One more time - it’s very important to make sure you query to get threads is correct.

The query is very flexible though. So I was able to use wildcards and change the to/from to also clean up noreply.github.com, googlegroups.com, and several others where I have accumulated mass amounts of junk over time.

As I said, I’m also impatient, so once I had these cleaned up, I made another pass through some of the worse offenders and added in:trash to the query to immediately clean out some messages from the trash.

All tidied up

In the end, my total Google account storage went down from about 12.78G to 11.8G. Not bad.

I’m guessing I can’t automate Google Photos clean up as well. At least not without learning some image recognition APIs and adding a lot of risk. So that’s a clean up for another day.

But with a few quick hacky scripts, I’m pretty happy with how much space I was able to free up.

November 2019 OpenStack Board Notes

2019-11-13T00:00:00+00:00

The Open Infrastructure Summit was held in mainland China for the first time the week of November 4th, 2019, in Shanghai. As usual, we took advantage of the opportunity of having so many members in one place by having a Board of Directors meeting on Sunday, November 3.

Attendance was a little lighter due to visa challenges, travel budgets, and other issues. But we still had a quorum with a lot of folks in the room, and I’m sure it was a nice change for our Chinese board members and others from the APAC region.

The original meeting agenda is published on the wiki as usual.

OSF Updates

Following the usual pattern, Jonathan Bryce kicked things off with an update of Foundation and project activity.

One interesting thing that really stood out to me, which Jonathan also shared the next day in the opening keynotes, as an analyst report putting OpenStack’s market at $7.7 billion in 2020. I am waiting for those slides to be published, but I think this really showed that despite the decrease in investment by companies in the development of OpenStack, its adoption and growth is stable and growing.

This was especially highlighted in China, with companies like China UnionPay, China Mobile, and other large companies from other industries increasing their use of OpenStack. And public clouds like Huawei and other local service providers basing their services on top of OpenStack.

I can definitely state from experience after that week, access to the typical big 3 public cloud providers in the US is a challenge through the Great Firewall. Being able to base your services on top of a borderless open source option like OpenStack is a great option with the current political pressures. A community-based solution, rather than a foreign tech company’s offerings, probably makes a lot of sense and is helping drive this adoption.

Of course, telecom adoption is still growing as well. I’m not as involved in that space, but it really seems like OpenStack is becoming the de facto standard for having a programmable infrastructure to base dynamic NFV solutions on top of, but directly with VMs and baremetal, and as a locally controlled platform to serve as the underlying infrastructure for Kubernetes.

Updates and Community Reports

StarlingX Progress Report

The StarlingX project has made a lot of progress over the last several months. They are getting closer and closer to the latest OpenStack code. They have been actively working on getting their custom changes merged upstream so they do not need to continue maintaining a fork. So far, they have been able to get a lot of changes in to various projects. They hope to eventually be able to just deploy standard OpenStack services configured to meet their needs, focusing instead on the services on top of OpenStack that make StarlingX attractive and a great solution for edge infrastructure.

Indian Community Update

Prakash Ramchandran gave an update on the various meetups and events being organized across India. This is a large market for OpenStack. Recently approved government initiatives could make this an ideal time to help nurture the Indian OpenStack community.

I’m glad to see all of the activity that Prakash has been helping support there. This is another region where I expect to see a lot of growth in OpenStack adoption.

Interop Working Group

Egle gave an update of the Interop WG activity and the second 2019 set of changes were approved. Nothing too exciting there, with just minor updates to the interop requirements.

The larger discussion was about the need and the health of the Interop WG. Chris Hoge was a very active contributor to this, but he recently left the OSF, and the OpenStack community, to pursue a different opportunty. Egle Sigler is really the only one left on the team, and she has shared that she would not be able to do much more with the group other than keeping the lights on.

This team is responsible for the guidelines that must be followed for someone to certify their service or distribution of OpenStack meets the minimum functionality requirements to be consistent with other OpenStack deployments. This is certification is needed to be able to use the OpenStack logo and be called “OpenStack Powered”.

I think there was pretty unanimous agreement that this kind of thing is still very important. Users need to be able to have a consistent user experience when moving between OpenStack-based clouds. Inconsistency would lead to unexpected behaviors or responses and a poor user experience.

For now it is a call for help and to raise awareness. It did make me think about how we’ve been able to decentralize some efforts within the community, like moving documentation into each teams repos rather than having a centralized docs team and docs repo. I wonder if we can put some of this work on the teams themselves to mark certain API calls as “core”, then some testing in place to ensure none of these set APIs are changed or start producing different results. Something to think about at least.

First Contact SIG Update

The First Contact SIG works on things to make getting involved in the community easier. They’ve done a lot of work in the past on training and contributor documentation. They’ve recently added a Contributing Organization Guide that is targeted at the organization management level to help them understand how they can make an impact and help their employees to be involved and productive.

That’s an issue we’ve had to varying degrees in the past. Companies have had good intentions of getting involved, but they are not always sure where to start. Or they task a few employees to contribute without a good plan on how or where to do so. I think it will be good having a place to direct these companies to, to help them understand how to work with OpenStack and an open source community.

Troila Gold Member Application

Troila is an IT services company in China that provides a cloud product based on OpenStack to their customers. They have been using OpenStack for some time and saw the value in becoming an OSF Gold level sponsor.

As part of the Member Committee, Rob Esker and I met with them the week prior to go over their application and answer any questions and give feedback. That preview was pretty good, and Rob and I only had minor suggestions for them to help highlight what they have been doing with OpenStack and what their future plans were.

They had taken these suggestions and made updates to their presentation, and I think they did a very nice job explaining their goals. There was some discussion and additional questions from the board, but after a quick executive session, we voted and approved Troila as the latest Gold member of the OpenStack Foundation.

Combined Leadership Meeting

The second half of the day was a joint session with the Board and the Technical Committees or Technical Steering Committees of the OpenStack, StarlingX, Airship, Kata, and Zuul projects. Each team gave a community update for their respective areas.

My biggest takeaway from this was that although we are unresources in some areas, we really do have a large and very active community of people that really care about the things they are working on. Seeing growing adoption for things like Kata Containers and Zuul is really exciting.

Next Meeting

The next meeting will be a conference call on December 10th. No word yet on the agenda for that, but I wouldn’t expect too much being so soon after Shanghai. I expect their will probably be some buzz about the annual elections coming up.

Once available, the agenda will be published to the usual spot.

I have the issue that I have been able to finish out my term because the rest of the board voted to allow me to do so as an exception to the two seat per company limit since I had rejoined Dell half way through the year. That won’t apply for the next election, so if the three of us from Dell all hope to continue, one of us isn’t going to be able to.

I’ve waffled on this a little, but at least right now, I do think I am going to run for election again. Prakash has been doing some great work with his participation in the India OpenStack community, so I will not feel too bad if I lose out to him. I do think I’ve been more integrated in the overall development community, so since an Individual Director is supposed to be a representative for the community, I do hope I can continue. That will be up to the broader community, so I am not going to worry about it. The community will be able to elect those they support, so no matter what it will be good.

Why is the Cinder mascot a horse?!

2019-11-12T00:00:00+00:00

I have to admit, I have to laugh to myself every time I see the Cinder mascot in a keynote presentation.

History (or, why the hell is that the Cinder mascot!)

The reason at least a few of us find it so funny is that it’s a bit of an inside joke.

Way back in the early days of Cinder, someone from Solidfire came up with a great looking cinder block logo for the project. It was along the style if the OpenStack logo at the time and was nice and recognizable.

Then around 2016, they decided it was time to refresh the OpenStack logo and make it look more modern and flat. Our old logo no longer matched the overall project, but we still loved it.

I did make an attempt to update it. I made a stylized version of the Cinder block logo using the new OpenStack logo as a basis for it. I really wish I could find it now, but I may have lost the image when I switched jobs. You may still see it on someone’s laptop - I had a very small batch of stickers made while I was still Cinder PTL.

It was soon after the OpenStack logo change that the Foundation decided to introduce mascots for each project. They were asking for each team to thing of an animal that they could identify with. It was supposed to be a fun exercise for the teams to be able to pick their own kind of logo, with graphic designers coming up with very high quality images.

The Cinder team didn’t really have an obvious animal. At least not as obvious as a Cinder block had been. It was during one of our midcycle meetups in Ft. Collins, Co while we were brainstorming that led to our horse.

Trying to think of something that would actually represent the team, we were talking over what Cinder actually was. We were mostly all from different storage vendors. We refer to the different storage devices that are used with Cinder as backends.

Backends are also what some call butts. Butts… asses. Donkeys are also called asses. Donkey!

One or two people on the team had cultural objects to having a donkey as a mascot. They didn’t think it was a good representation of our project. So we compromised with going with a horse.

So we asked for a horse to be our mascot. The initial design they came up with was a Ferrari looking stallion. Way to sporty and fierce for our team. Even though the OpenStack Foundation has actually published it and even created some stickers, we explained our, erm… thought process… behind coming up with the horse in the first place. The design team was great, and went back to the drawing board. The result is the back-end view of the horse that we have today. They even worked a little ‘C’ into the swish of the horse’s tail.

So that’s the story behind the Cinder logo. It’s just because we’re all a bunch of backends.

October 2019 OpenStack Board Notes

2019-10-22T00:00:00+00:00

Another OpenStack Foundation Board of Directors meeting was held October 22, 2019. This meeting was added primarily as to discuss the Airship’s request to for confirmation to become an official project.

The meeting agenda is published on the wiki.

OSF Updates

Jonathan Bryce gave a quick update on the OpenStack Train release that went out last week. The number of contributors, variety of companies, and overall commit numbers were pretty impressive. There were over 25,500 merged commits in Train, with 1,125 unique developers from 165 different organizations. With commits over the last cycle, OpenStack is still one of the top three active open source projects out there, after the Linux kernel and Chromium.

Jonathan also reiterated that the event structure will be different starting in 2020. The first major event planned is in Vancouver, June 8. This will be more of a collaborative event, so expect the format to be different than past Summits. I’m thinking more Project Teams Gathering than Summit.

Airship Confirmation

Matt McEuen, Alex Hughes, Kaspar Skels, and Jay Ahn went through the Airship Confirmation presentation and answered questions about the project and their roadmap. Overall, really pretty impressive what the Airship community has been able to accomplish so far.

The Airship mission statement is:

Openly collaborate across a divers, global community to provide and integrate a collection of loosely coupled but interoperable, open source tools taht declaratively automates cloud lifecycle management.

They started work inside of openstack-helm and kept to the OpenStack community Four Opens right from the start.

Project Diversity

The project was started by AT&T, so there is still a lot of work being done (code reviews, commits, etc.) from the one company, but the trend over the last couple of years has been really good, trending towards more and more contributor diversity.

They also have good policies in place to make sure the Technical Committee and Working Committee have no more than two members from the same company. Great to see this policy in place to really encourage more diversity in the spots where overall project decisions are made. Kudos to the AT&T folks for not only getting things started, but driving a lot of change while still actively encouraging others so it is not a one company show. It can be hard for some companies to realize that giving up absolute control is a good thing, especially when it comes to an open source community.

Community Feedback

Part of the confirmation process is to make sure the existing OSF projects are comfortable with the new project. There was feedback from the Zuul project and from the OpenStack TC. Rico Lin went through the TC feedback in the meeting. Only minor questions or concerns were raised there, and Matt was able to respond to most of them in the meeting. He did state he would respond to the mailing list so there was a record there of the responses.

Licensing

Really the only point of concern was raised at the end. One difference between Airship and other OpenStack projects is that it is written in Go. Go has a great system built in to be able to easily use modules written by others. But that led to the question of licensing.

The OSF bylaws state:

The Board of Directors may approve a license for an Open Infrastructure Project other than Apache License 2.0, but such license must be a license approved by the Open Source Initiative at the date of adoption of such license.

The Airship code itself is Apache 2.0. But there isn’t anything done today to vet the dependencies that are pulled in to actually compile the project. The concern is the copyleft licenses usually have provisions that if they are pulled in and linked to non-copyleft code, it then makes that code fall under the copyleft requirements. So the only concern was that it just wasn’t known what the effective license of the project is today based on what is being pulled in.

It can be a very tricky area that definitely requires involvement of lawyers that understand copyright law and open source licensing. Luckily it wasn’t a show stopper. We moved to add the project and have them work with OSF legal to better understand the licensing impacts and resolve any concerns by using different dependencies if any are found to be licensed with something that would impose copyleft into Airship. The board unanimously voted in favor of Airship becoming a fully official Open Infrastructure Project.

Next Meeting

The next OSF board meeting will take place November 3rd, in Shanghai, the day before the Open Infrastructure Summit.

September 2019 OpenStack Board Notes

2019-10-17T00:00:00+00:00

There was another OpenStack Foundation Board of Directors conference call on September 10, 2019. There were a couple of significant updates during this call. Well, at least one significant for the community, and one that was significant to me (more details below).

In case this is your first time reading my BoD updates, just a reminder that upcoming and past OSF board meeting information is published on the wiki and the meetings are open to everyone. Occasionally there is a need to have a private, board member only portion of the call to go over any legal affairs that can’t be discussed publicly, but that should be a rare occasion.

September 10, 2019 OpenStack Foundation Board Meeting

The original agenda can be found here. Usually there are official and unofficial notes sent out, but at least at this time, it doesn’t appear Jonathan has been able to get to that. Watch for that to show up on the wiki page referenced in the previous section.

Director Changes

There were a couple of changes in the assigned Platinum Director seats. The Platinum level sponsors are the only seats on the board that are guaranteed to the sponsor and allows them to assign a Director. So no change in sponsorships at this point, just a couple of internal personel changes that led to these changes.

With all the churn and resulting separation of Futurewei in the US from the rest of Huawei, their chair seat was moved over to Fred Li. I worked with Fred quite a bit during my time with the company. He’s a great guy and has put in a lot of work, mostly behind the scenes, to support OpenStack. Really happy to be able to work with him again. Anni has also done a lot over the years, so sad to see her go. I’m sure she will be quite busy on new things though.

On the Red Hat side, Mark McLoughlin has transitioned out, handing things over to Daniel Becker. It sounds like with the internal structure at Red Hat, Daniel is now the better representative for the OpenStack Foundation. I personally didn’t get a lot of opportunity to work with Mark, but I know he has been around for a long time and has done a lot of great things, so I’m a little sad to see him go. But also looking forward to working with Daniel.

Director Diversity Waiver

This was the significant topic to me, because, well… it was about me.

In June I switched employers, going back to Dell EMC. So far, I’ve been very happy, and it feels like I’ve gone bake home with the 14+ years between Compellent and Dell that I had prior to joining Huawei. Not that my time with Huawei wasn’t great. I think I learned a lot and had some opportunities to do things that I hadn’t done before, so no regrets.

But the catch with my going back to Dell was that they already have a Gold sponsor seat with Arkady Kanevsky and a community spot with Prakash Ramchandran.

The OpenStack Foundation Bylaws have a section (4.17) on Director Diversity. This clause limits the number of directors that can be affiliated with the same corporate entity to two. So even though Prakash and I are Individual Members (which means we are there as representatives of the community, not as representatives of our company), my move to Dell now violated that clause.

I think this was added to the bylaws back in the days where there were a few large corporate sponsors that had large teams of people dedicated to working on OpenStack. It was a safeguard to ensure no one company could overrun the Foundation based solely on their sheer number of people involved. That’s not quite as big of an issue today, but I do still think it makes sense. It is a very good thing to make sure any group like this has a diversity of people and viewpoints.

The bylaws actually explicitly state what should happen in my situation too - Article 4.17(d) states:

If a director who is an individual becomes Affiliated during his or her term and such Affiliation violates the Director Diversity Requirement, such individual shall resign as a director.

As such, I really should have stepped down on moving to Dell.

But luckily for me, there is also a provision called out in 4.17(e):

A violation of the Director Diversity Requirement may be waived by a vote of two thirds of the Board of Directors (not including the directors who are Affiliated)

This meant that 2/3 of the Board members present, not including any of us from Dell, would have to vote in favor of allowing me to continue out my term. If less than that were in favor, then I would need to step down. And presumably there would just be an open board seat for the rest of the term until elections are held again.

There was brief discussion, but I was very happy that everyone present did vote in favor of allowing me to continue out my term. I kind of feel like I should have stepped out during this portion of the call to make sure no one felt pressure by not wanting to say no in my presence, but hopefully that wasn’t the case for anyone. It was really nice get these votes, and some really good back channel support from non-board attendees listening in on the call.

What can I say - compliments and positive reinforcement go far with me. :)

So I’m happy to say I will at least be able to finish out my term for the rest of 2019. I will have to see about 2020. I don’t believe Arkady nor Prakash are planning on going anywhere, so we may need to have some internal discussions about the next election. Or, probably better, leave it up to the community to decide who they would like representing them for the Individual Director seats. Prakash has been doing a lot of great work for the India community, so if it came down to it and I lost to him, I would be just fine with that.

OW2 Associate Membership

Thierry then presented a proposal to join OW2 as an Assocaite Member. OW2 is “an independent, global, open-source software community”. So what does that mean? Basically, like the Open Source Initiative and others, they are a group of like-minded individuals, companies, and foundations that work together to support and further open source.

We (OpenStack) have actually worked with them for some time, but we had never officially joined as an Associate Member. There is no fee to join at this level, and it is really just formalizing that we are supportive of OW2’s efforts and willing to work with them and the members to help support their goals.

They have been in support of OpenStack and open infrastructure for years, so it was great to approve this effort. We are now listed as one of their Associate Members.

Interop WG Guidelines 2019.06

Egle moved to have the board approve the 2019.06 guidelines. We had held an email vote for this approval, but since we did not get reponses from every Directory, we now performed a vote in-meeting to record the voting. All present were in favor.

The interop guidelines are a way to make sure all OpenStack deployments conform to a base set of requirements. This makes sure that an end user of an OpenStack cloud has at least some level of assurance that they can move from one cloud to another and not getting a wildly different user experience. The work of the Interop Working Group has been very important to ensuring this stability and helping the ecosystem around OpenStack grow.

Miscellaneous

Prakash gave a quick update on the meetups and mini-Summits being organized in India. Sounds like a lot of really good activity happening in various regions. It’s great to see this community being supported and growing.

Alan also made a call for volunteers for the Finance and Membership committees. I had tried to get involved earlier in the year, but I think due to timing there really just wasn’t much going on at the time. With the next election coming up, and some changes in sponsors, now is actually a good time for the Membership Committee to have some more attention. I’ve joined Rob Esker to help review any new Platinum and Gold memberships. Sounds like we will have at least one new one of those coming up soon.

Summit Events

It wasn’t really an agenda topic for this Board Meeting, but I do think it’s worth pointing out here that the proposed changes to the structure of our yearly events have gone through and 2020 will start to diverge from the typical pattern we have had so far of holding to major Summits per year.

Erin Disney sent out a post about these changes to the mailing list. We will have a smaller event focused on collaboration in the spring, then a larger Summit (or Summit-like) event in later in the year.

With the maturity of OpenStack and where we are today, I really think this makes a lot more sense. There simply isn’t enough big new functionality and news coming out of the community today to justify two large marketing focused events like the Summit per year. What we really need now is to foster the environment to make sure the developers, operators, and others that are working on implementing new functionality and fixing bugs have the time and venue they need to work together and get things done. Having these smaller events and supporting more things like the regional Open Infrastructure Days will hopefully help keep that collaboration going and allow us to focus on the things that we need to do.

And the next event will be in beautiful Vancouver again, so that’s a plus!

April 2019 OpenStack Board Notes

2019-04-08T00:00:00+00:00

These are just some of my notes from the OpenStack Foundation Board of Directors meeting that took place on April 8, 2019.

Upcoming and past OSF board meeting information is published on the wiki and the meetings are open to everyone. Occasionally there is a need to have a private, board member only portion of the call to go over any legal affairs that can’t be discussed publicly, but that should be a rare occasion.

This meeting was added to discuss some of the incubating project confirmations ahead of our next face-to-face “joint leadership” meeting with the Technical Committee and User Committee on April 28, the Sunday prior to the next Open Infrastructure Summit in Denver, Co.

April 8, 2019 OpenStack Foundation Board Meeting

The original agenda can be found here and Jonathan Bryce sent out some unofficial minutes to the Foundation mailing list. The April 8th notes can be found here.

Project Confirmation

After the typical roll-call and approving prior meeting minutes administrative activities, our first order of business for the day was to review proposals from the two pilot projects that are ready for confirmation to become full OpenStack Foundation top level projects: Zuul and Kata Containers.

This was initially supposed to be a walk through of the presentations with part of the goal of seeing what other information we might need to be able to fully evaluate them before making anything official. At least in the case of Kata Containers, we felt there really wasn’t anything more needed. So after some quick discussion of whether to defer until the meeting in Denver, we decided to hold the vote today. More details below.

These presentations by each team were driven by ansering the factors called out in the OSF Project Confirmation Guidelines. This was our first time going through a real world case of answering these, and I think they held up well and ended up being a good way to cover most areas of concern and guide the discussions.

Kata Containers

Eric Ernst from Intel presented for Kata Containers.

Mission Statement

Openly collaborate across a diverse, global community to define and implement secure, versatile container solutions that combine the benefits of virtualization with the performance and ease of containers.

Basically, Kata is a container runtime that leverages lightweight virtual machines to provide the isolation and security of VMs with the speed and flexibility of containers.

Governance

The Kata Containers community is made up of Contributors and Maintainers, with an Architecture Committee responsible for overall architectural decisions and making final decisions if Maintainers disagree. This seems roughly equivalent to OpenStack’s Contributors and Core Reviewers, with the Architecture Committee equivalent to what some have argued the OpenStack Technical Committee should really be.

Really great to hear that they do have policies in place that no more than two members of the AC can be from the same company and that they have already been holding elections every six months using the same condorcet method CIVS voting as used in the OpenStack community.

Technical Best Practices

Kata has documentation available and it is considered an ongoing and active focus.

Code contributed to the project goes through peer code review before being accepted.

Test and CI/CD is enforced on code changes to help ensure quality.

Bug/issue tracking is in place and security VMT is in place.

Open Collaboration

I was really glad to see the Kata Community is following the “4 Opens”:

Open Source

Open Design

Open Development

Open Community

I think this is really the cornerstone of what has united the OpenStack Community, so my concern with expanding the umbrella of the OpenStack Foundation was making sure the new projects coming on also believed in these principles. It’s really great to see the Kata Community believes in them too and considers all four as important elements to an open source project and community.

Active Engagement

Big kudos to the team for the level of involvement they’ve had with trying to get out and get active engagement from contributors and users. They’ve had a presence at the past OpenStack Summits and will be at the upcoming Open Infrastructure Summit. They will also be holding sessions at the Project Teams Gathering following the Summit in Denver.

They’ve also made an effort to get out to other communities. They’ve been attending OpenStack and Open Infra Days events, but have also been active at Kubecon, KVM-Forum, Open Source Summit, and DevSecCon. All great areas for this type of project.

Voting

All questions were answer openly and clearly, and there really didn’t seem to be much reason to wait to do a vote. Concern was raised by some board members that we had said we would be doing the vote at the end of the month so we should stick to what we had said, but after a quick poll of the board members it was decided to move ahead with the approval vote.

With one abstaining (due to the change in our stated voting plan) we did approve the Kata Containers confirmation. Kata is now an official full top level project of the OpenStack Foundation.

Zuul

The Zuul CI project grew out of (and in many ways helped shape) the OpenStack project and community, so this seemed like it would be even less of a discussion.

Presentation

Monty Taylor presented the slides that the Zuul team has made available. I believe the plan is to keep that available, so I won’t go over each section here. I think it’s enough to say, all the areas that the board is concerned about for confirming a project have been followed and shaped by the Zuul team well before we even starting talking about other top level projects.

Voting

Voting did hit a snag for this project though. One piece of Zuul, the zuul-preview component does get compiled with some GPLv3 libraries. The OpenStack Governance Bylaws do specifically call out the Apache 2.0 license.

I think it may be a matter of properly phrasing things, but since no one had a clear answer if the use of GPLv3 was acceptable in this situation, we plan on working through the legal aspects of this and regroup for further discussion at the face-to-face meeting in Denver.

Bylaws Change

A potential change in the wording of another bit of the bylaws was brought up, with the plan to actual vote on the change in Denver.

4.16 Open Meetings and Records. Except as necessary to protect attorney-client privilege, sensitive personnel information, discuss the candidacy of potential Gold Member and Platinum Members, and discuss the review and approval of Open Infrastructure Projects, the Board of Directors shall: (i) permit observation of its meetings by Members via remote teleconference or other electronic means, and (ii) publish the Board of Directors minutes and make available to any Member on request other information and records of the Foundation as required by Delaware Corporate Law.

This was in reaction to past meetings where it was brought up that some board members may have some reasons where they can’t speak publicly about a project or backing company being considered and would need a private “closed door” session to be able to raise their concerns.

Setting aside for now that I’m against doing any of the new project evaluations behind closed doors, I did raise on the mailing list before the meeting that I would really rather this be reworded to make sure it is clear that these meetings would only be an exception basis, and as much as possible all discussions about new projects should be done in the open for all to see. The way this is worded now, to me at least, reads like these meetings are expected to be done in private. I know that is not the intent of everyone involved right now, but I would rather have that intent made clear in the way this is added in the bylaws versus just leaving it open for interpretation.

I’m sure there will be plenty more discussion around the wording, and even the for this before we meet in Denver.

March 2019 OpenStack Board Notes

2019-03-06T00:00:00+00:00

This is the second report in a hopefully ongoing series of notes capturing OpenStack Foundation Board Meeting activities.

As a reminder (or in case you weren’t aware) the planned OSF board meetings are published on the wiki and are open to everyone. Occasionally there is a need to have a private, board member only portion of the call to go over any legal affairs that can’t be discussed publicly, but that should be a rare occasion.

I would encourage anyone interested to listen in. Or join us at the next face-to-face “joint leadership” meeting with the Technical Committee and User Committee on April 28, the Sunday prior to the next Open Infrastructure Summit in Denver, Co.

March 5, 2019 OpenStack Foundation Board Meeting

The original agenda can be found here and the official minutes are here. Jonathan Bryce also usually sends out unofficial minutes to Foundation mailing list. The March 5th notes can be found here.

Board Member Changes

The Platinum and Gold board member positions are tied to the sponsor organizations, with the Platinum members being appointed and the Gold members being selected as part of an election amongst that group.

Due to shifting job responsibilities, career moves, etc., representatives from this group do change from time to time. Brian Stein (Rackspace) and Mark Baker (Canonical) have left the board, replaced by Andy Cathrow and Ryan Beisner, respectively.

Confirmation Guidelines

Allison Randall has been leading the effort of formalizing confirmation guidelines for new projects coming in under OpenStack Foundation governance. The draft of these guidelines have been available for some time, so this was more of a final review to see if there were any further details that needed to be worked through.

Personally, I find section 3 (“Technical best practices”) to be a little vague in practice, but not enough to raise it as an issue in the meeting. These are guidelines afterall, not a legal rubric.

I am very happy with the detail in section 4 (“Open collaboration”). The items listed under here were the main points of concern I had with how new projects will be handled. I believe things like following the Four Opens and adhering to the community code of conduct are very important to make sure there is some sort of cohesion between the different groups that will make up the greater community. I’m also happy to see having an OSI license explicitly called out.

The main conversation around this topic ended up being the need (or not) to have non-public discussions about incoming projects. There were some strong opinions as to whether it was even appropriate for an open source organization such as ours should even have private board meetings to discuss these things and whether that would cause the appearance of making decisions on new projects behind closed doors.

Mark Radcliffe’s main point was there could be times where legal reasons would cause the need. The good example I saw was if a director worked for a company that somehow had a legal restriction that would prevent them from making public comments about a project being led by a competing organization.

Then further down the route of needing private board meetings, would it be better to have them for every project so it’s not as obvious that there are some concerns when they actually are needed for a new project.

I think it was Mark McLoughlin that raised the point that we do reserve the option to have an exclusive portion of our regular board meetings, so we could be discrete by letting Alan Clark know that we would like to use that time at the next board meeting to have a private discussion without making it too obvious that there are concerns about a project.

I could see some of the legal arguments for maybe needing this, but ultimately I think we need to stay open in these discussions. By being part of the board, we are taking on the obligation of publicly discussing matters of the OSF. If someone has a legal reason to not discuss something in the open, then I think it’s probably better they either recuse themselves from the discussion or have a direct conversation with the team or Alan to raise their concerns.

We are pushing for the Four Opens for the community. I believe the Board needs to operate as openly as possible too.

We will be planning another meeting in April to give everyone time to think through some of the concerns and then hopefully agree on how this will be handled going forward.

The good thing is that the confirmation guidelines themselves were approved and we now have something in place as we evaluate new OSF projects.

Compensation Committee Update

Alan presented the current draft of the compensation committee’s work on setting goals for the OpenStack Foundation staff for 2019. I am not able to find a public reference to that at the moment, but I will see about posting a future update.

Basically, this sets the goals for Jonathon Bryce for directing Foundation staff activities for the year. It includes things like the new project confirmations, promotion of OpenStack activities, and other efforts for the staff to work towards.

India Events

Prakash Ramachandran raised this topic to get some awareness of efforts he is leading to promote OpenStack and Open Infra events in India, and the desire to at least have a discussion about a potential future Open Infrastructure Summit in India in the future.

Unfortunately we ran out of time. I would have liked to have had more discussion on this topic. We have a very large contributor base in India and many Indian companies using OpenStack. I think it would be great to help connect with this community by having a Summit there and I was glad to hear about some of the smaller events Prakash and team have been organizing in different regions.

(side note: I had a great experience going to an OpenStack Days event in Bangalore a few years ago.)

Hopefully we will hear more on this in the future.

Jan 2019 OpenStack Board Notes

2019-02-11T00:00:00+00:00

Chris Dent used to post regular, semi-subjective, takes on what was going on in the OpenStack Technical Committee. Even though I was there for most of the conversations that were being recapped, I found these very valuable. There were many times that multiple things were discussed over the course of the week, often overlapping, and usually competing for metal attention. So having these regular recaps helped me to remember and keep up on what was going on. I also found the subjective nature useful over “just the facts” summaries to be able to understand some perspectives on topics that I sometimes agreed with, sometimes didn’t, and often just hadn’t taken into consideration.

That was all while being actively involved in the TC. So I can imagine if there was someone interested in what was going on, but not able to devote the significant time it takes to read IRC logs, mailing list posts, and patch reviews that it takes to really stay on top of everything, these kinds of things are really the best way for them to be able to get that type of information.

This is my first attempt to take that kind of communication and apply it to the OpenStack Foundation Board meetings. The Board of Directors meet several times a year, usually via conference calls with a few face to face meetings where possible around Summits and other events. Hopefully these will be useful for anyone interested in what is happening at the board level. I’m also hoping they help me be better about taking notes and keeping track of things, so hopefully it’s a win-win.

January 29, 2019 OpenStack Foundation Board Meeting

The original agenda can be found here and the official minutes will be posted here. Jonathan Bryce also usually sends out unofficial minutes to Foundation mailing list. The January 29th notes can be found here.

Administrivia

Alan Clark started things off with the typical procedural stuff for meetings like this. Roll call was taken to make sure there was quorum. Minutes from the last board meeting in December were voted on and officially approved.

These meetings just recently switched from using WebEx to using Zoom. Apparently, as a result of that, the normal meeting reminders that folks were used to were not sent out, so there were a few absent and late. But I think overall there was a pretty good showing for the 28 board members. I believe there were only six board members not in attendance, which considering time zones, seems reasonable.

Alan thanked the outgoing board members for their time on the board. There were five of us, across Platinum, Gold, and Individual members, that were new to the board and we all had a few minutes to introduce ourselves and give a little background before moving on to business.

Policy Reminders

Especially useful for the new folks like me, there was a reminder about some of the policies that board members need to follow. I was aware of some, but definitely not all of these. I think it’s useful to include here:

Reading the Transparency policy was useful, and I especially liked seeing there is a heading title “General Policy Favoring Transparency”.

Meeting Schedule

Two face to face meetings are planned this year, one before the next Open Infrastructure Summit in Denvery in April, and another date to be determined before the following Summit in Shanghai, China in early November.

Speaking of which, up until that point we knew the Summit was planned in China but no official location was announced. It is now official that the Open Infrastructure Summit China will be held the week of November 4 in Shanghai.

Committee Updates

There are various committees within the board to work on different areas. The list of committees can be found on the Foundation wiki.

Most of the discussion was just making everyone aware of these. There are some listed on the wiki that are just there for historical purposes.

Since financial issues have a lot of impact on the community, especially now that there are less sponsors than there were a few years ago, I have decided to join the Compensation Committee. There are a few others that I find interesting and think are important, but I will wait a little bit before signing up for more. I know I have a tendency to want to sign up for anything I think I can help with without really thinking too much about the time commitment, so I am trying to be better about raising my hand too much.

OpenStack Foundation Pilot Project Guidelines

Allison Randall have a report on the effort to coming up with a set of written guidelines for new projects coming in as pilot projects under the OpenStack Foundation.

As the Foundation expands its scope to include more project beyond OpenStack to support “open infrastructure”, I think it’s important that we are careful about what we include to keep true to our existing community and our core identity. This group is working on writing a set of guidelines to help ensure that happens.

The current draft guidelines can be found here.

Some I would like to see a little more specific or reworded to be less subjective (following “Technical best practices”?) but I am happy to see things called out like “Open collaboration”. I am also a little concerned about how open governance is left.

To me, the Four Opens are what really defined the OpenStack community compared to other open source environments I had seen. I will have a hard time feeling really comfortable with adding any new projects that do not at least strive towards following the four opens.

Foundation Update

Jonathan and team wrapped thing up with a staff update. The slides can be found here.

It was great to see that there has actually been a 33% increase in community membership over the last year. But seeing the activity in the projects, this tells me that more and more involvement is casual or part-time. We have been working on making things more welcoming to new contributors and talking about ways to make contributing easier for those that aren’t spending a significant part of their work week on OpenStack and related projects. I think these efforts are very important and will be critical to our ability to getting more done going forward.

Following on from Allison’s update, there was a large part about the expanding role of the Foundation and the projects currently in the Pilot phase. Right now, these are Airship, Kata Containers, StarlingX, and Zuul.

There was some good coverage about where the Foundation can play a role in an expanded role of promoting open infrastructure. The main goals for this year were reported as strengtheing OpenStack (branch and community), helping expand market opportunities for open infrastructure, and evolving the business model of the Foundation to be this broader-scoped entity.

I’ll probably have more to say on some of those soon, but overall I think it was a good update and, all things considered, I think we are on the right track and at least paying attention to the things we need to going forward.

Gofish - A golang client library for Redfish and Swordfish

2019-02-03T00:00:00+00:00

One of the things I have been looking in to for the OpenSDS project is adding support for the Swordfish API standard. I believe by OpenSDS providing a standardized API it could both make it attractive for storage vendors to natively integrate with OpenSDS - with one integration they would pick up support for CSI, Cinder, Swordfish, and other integration points - as well as make OpenSDS an interesting option for those needing to manage a heterogenous mix of storage devices in their data center in a common, single pane of glass, way.

We are still looking at how this could be integrated and how much focus it should have. But it became clear to me that we would need a good way to exercise this API if/when it is in place.

We are also considering southbound integration in OpenSDS to be able to manage Swordfish-enabled storage. This would enable OpenSDS support for storage without needing to write any custom code if they have already made the investment in exposing a Swordfish API. If we were to do that, we would need a client library that would allow us to easily interact with these devices.

So between the two approaches being considered, along with wanting to get a little more time and experience with golang, I decided to start implementation on an open library for Redfish and Swordfish. I’ve just pushed up a very rudementary start with the Gofish library.

What is Swordfish

The DMTF released the first Redfish standard in 2015. The goal of Redfish was to replace IPMI and other proprietary APIs for managing data center devices with a simple, standard, RESTful API that would provide consistency and ease of use across physical, hybrid, and virtual IT devices.

For years, the Storage Network Industry Association (SNIA) had been working on SMI-S as a standard storage management protocol. There was industry adoption, but there was still a general sense that SMI-S was too big and too complex for both implementors and consumers of the API.

SNIA recognized the simplicity and ease of use of the Redfish specification and chose to extend the DMTF spec with storage related objects and capabilities. These additions can be seen (at a high level) with the purple objects in this model from a presentation by Richelle Ahlvers, the chair of the Scalable Storage Management Technical Working Group (SSM TWG):

Introducing Gofish

To start, I just used the API responses from the Swordfish API emulator as a reference to get a very basic client library working which I have called Gofish. Cause I’m just so clever and witty like that.

This is very, very rudimentary at this point. There is no authentication mechanism yet, something that any real Redfish or Swordfish provider will surely require. The object model is also very limited and incomplete. Luckily all of the Redfish and Swordfish schemas are published in easy to consume yaml or json formats that should make it easy to script up the generation of the rest of the schema.

Even in its basic state, I did want to get this out there in case anyone else is interested. I’ve published this under the Apache 2 license in a public GitHub repo, so anyone that is interested in contributing, feel free to propose pull requests and try out the code.

I am hoping to work on this as time permits to get it more full featured and robust. I am limited by my access to Redfish and Sworfish enabled devices, so I may be constrained until I can track down something more than the emulator to use for testing. But hopefully this project evolves into something useful as more devices implement this support and more management tools are developed to take advantage of it.

Running for the OpenStack Board of Directors (Take 2)

2019-01-03T00:00:00+00:00

Just a year ago I had put in my name for the OpenStack Board of Directors Individual Director election. The board is divided into 8 appointed directors from Platinum member companies, 8 directors elected from within the Gold member companies, and 8 individual directors from the community.

I think last year’s election was close, or at least I fared better than I had feared I would, so I am giving it another shot this year.

I was really on the fence about it in the months leading up to the nomination period. This post is a chance for me to jot down some of my thoughts that led me to decide it was the right thing to do.

The Role of Individual Members

The individual directors are members of the Foundation and are there to represent the members of the Foundation. Foundation membership is open to all, so I see this role as being a voice of the community within the Board governance - something that I personally find really valuable and necessary for something like a non-profit, open source community Board of Directors.

As part of my involvement in the OpenStack Technical Committee over the last couple of years, one of the things I have tried to encourage is more frequent and open communication between the Board and the TC. Only through good communication between these governance groups can we really be most effective and learn from each other to best support the goals of the Foundation and the needs of the community that has grown around OpenStack, open infrastructure, and those that care about the health and availability of open options.

I feel the need to preface the next part with saying there are some really great folks on the Board of Directors now that I truly believe are doing what they think is best to support the Foundation’s mission. These folks are taking time out of their busy schedules to be a part of this and give their knowledge and expertise to help guide the Foundation. Nothing I say here should be taken as a criticism for any of the decisions that have been made or as being directed by anyone that is now or has been a part of making OpenStack what it is today.

Now in that context, here’s where I see the need for folks like me to get involved.

For some board directors, the entirety of their participation in the OpenStack community is calling in to board meetings once a quarter. Some I’m sure are actively involved in their company’s interaction with their customers and the real world issues of trying to take the output of all of this activity and making it into something that can be packaged and supported and used to address their customer’s needs. Which is HUGELY important.

But to really understand how to shape things and make things better, I think it’s also very important to understand the development community and the challenges, needs, and pressures happening there to make the right long term direction choices.

That’s the role the individual directors fill, and these eight people need to be able to bridge that gap between knowing the right things for the companies involved, the operators trying to run the software, and the users that ultimately are trying to do something bigger and more important than whatever is used to enable their infrastructure to support their activities.

My Goals

Unlike company Board of Directors, open source foundation board effectiveness is limited to their influence within their own companies and making the case to others individuals and companies involved. Some of this is through financial direction of the foundation, and some of it is by being able to bring in their expertise and convince others whether certain things are the right thing to focus on for the long term success of the community.

Over the last several years I have worked in various areas of the community. I’ve been a PTL of a major component of most OpenStack clouds. I’m a core member of a few projects I feel are important to the overal success and health of the community. I’ve participated on the Technical Committee since spring of 2017. I’ve also spent time pushing a broom and taking care of some areas that no one notices until they start to smell.

I think whoever gets elected, they need to have that experience and be involved in the development of the code to be able to provide input on what is happening and how the board decisions could impact them.

I’ve also been a big supporter of the Ops Meetups, being involved in helping organize those events and participating by presenting, moderating, or helping to organize to be a productive forum for operators to share best practice and war stories and build this also important part of the community that we need to be a thriving group. As someone that has moved from more operations to more development, I have learned a tremendous amount from these folks and am in awe at the things they’ve been able to do and their dedication to OpenStack and willingness to share and learn and grow from each other.

I’ve also had the opportunity lately to get out and speak to customers. Some have been great OpenStack supporters. But perhaps the more important ones for me has been speaking to the ones that are not. It’s always hard to hear why something you’ve worked on for years isn’t good enough, or easy enough, or featureful enough to meet someone’s needs. But it’s also tremendously important to hear those things.

So my plan for being on the board is to be able to take my experiences as a developer within the community, a supporter of operators and users, and someone willing to process the criticisms of the software and feed those into the efforts and decisions made by the Board. Regardless of who is elected, these are all critical things that need a voice in those discussions.

Board Election Candidates

Like last time, the good news is there are a lot of really good candidates who have been willing to step up to this role.

Some definitely have more time spent in the community. Some likely have more experience operating an OpenStack cloud or running open infrastructure. Many probably have resumes that would put mine to shame. But I do hope I get the opportunity to be one of the eight individual directors. I do think I have a unique and important enough experience to provide an important voice within the OpenStack Foundation Board of Directors.

Please watch for ballots sent out when the election opens up January 14th. Read the candidate profiles and vote for whichever ones you think will be the right individual voices on the Board. The important thing is to participate and provide your vote to make sure we have a strong and health Board.