GroupVitals System Status

If there are any major issues with our system, we'll post them here πŸ‘

 Everything is Good πŸ‘πŸ»

Past Status Updates

🏁 Email Downtime

1/11/19 @ 6 AM CT

Our email system is back up and all emails that were supposed to go about between midnight and 6am have been sent.

1/7/19 @ 11 AM CT

We wanted to let you know that GroupVitals' email system will be down for maintenance this Saturday, January 11th, 2020. You can still login to GroupVitals and send emails on Saturday. They just won't be sent to the recipients until the system is back online. 

We'll do our best to get every automated and manual email sent that day to go out as soon as the email system is back up, or Sunday at the latest. If you have planned on sending any emails Saturday, you may want to send them before then.

FYI - We use MailChimp's enterprise Mandrill service to send emails with GroupVitals. Mandrill recently informed us that they are having this downtime to make system updates. This downtime is out of our control and something they've never done before with us. We're doing our best to roll with it and make some updates to our software so that your Saturday emails will get sent out as soon as Mandrill gets their email system back online.

🏁 Attendance Issue

5/16/19 @ 5 PM CT
At this point, all but one church has finished re-syncing data between GroupVitals and F1. Any missing attendance in F1 should also be showing up in F1 now. If you're seeing anything that isn't syncing or matching up, please contact me - and we'll get it all worked out πŸ™I'm sorry for the troubles here and I thank you guys for your patience! - Tony

5/15/19 @ 9:30 AM CT
We're about 90% caught up with syncing all accounts with F1. We have just a few accounts with very large databases that we're pulling in updated data from their F1 account. We expect this to be done by the end of the day. After that, 100% of accounts will be back in full operation with our F1 sync πŸ‘

5/14/19 @ 12:30 PM CT

We're just about done with getting F1 back to syncing with GroupVitals. We've pushed over to F1 all the changes made in GroupVitals. Now, we're pulling over all the updates that had been made to people and groups while our integration with F1 was down. We expect this to be done by the end of today. After that, everything should be in sync and the nightly F1 sync will be turned back on and everything full operational.

5/13/19 @ 2:30 PM CT

We've made a lot of progress over the weekend getting all of the F1 <> GroupVitals sync data in order. We've also had to write some custom code/scripts in order to complete this project of updating F1 accounts with the correct data. We're still in the process of comparing records to see which one is the correct one to update in either F1 or GroupVitals. Once that's done, we'll be able to begin the process of updating accounts. At this rate, we're hopeful we'll be all done by this Tuesday.

5/10/19 @ 10 AM CT

I wanted to let you know that we're currently experiencing some issues with FellowshipOne. The nightly sync is currently down, so any changes made in F1 are not being updated in GroupVitals right now. However, any changes made in GroupVitals are currently being sent to F1. We're working hard to getting the nightly sync back up and running by May 13th or 14th.

What happened:
On May 9th, we discovered that between April 20th and May 9th, any changes made in GroupVitals were not sent to F1. For some churches, it started on April 20th and for others, it started on May 9th, which is what made this issue hard to catch. The great news is that we have a record of all the updates that were not sent to your F1 account during this down period. The reason why the sync stopped working is that each month, F1 requires us to renew a special "token" for each church (kinda like a password) in order to keep syncing data with the church. These tokens expired and were not automatically renewed by our system as usual. There was an error in our code that prevented the token renewing from happening (that's been fixed now). So once a token expires, updates made in GroupVitals or F1 are prevented from transferring from one to the other. We've already renewed every church's token, so that's how you can currently make changes in GroupVitals and those will immediately be sent to F1.

What we're doing:
While the nightly sync with F1 is paused, we're going through our logs for each church to determine when their sync stopped, what information needs to be updated in F1 and what information needs to be updated in GroupVitals (based upon changes made in F1). Once we determine this information, we'll then proceed to send either send updates made to people and groups to F1 or pull updates from F1 to GroupVitals. It will all depend upon where and when the information was last updated for a person or group.  Once we get all the data updated and in sync, we'll be able to flip on the switch for the nightly sync with F1.

What you should do:
Please avoid performing a manual sync with F1 via the add-ons section (it won't really work). If possible, avoid updating anything related to a group within F1. We know that people will be updated within F1, so you/other staff can continue updating and adding people records in F1 - we'll bring over all those updates once we sort through the data and launch our fix.

I'll continue to post updates to this page.

If you have any questions, don't hesitate to reach out:

🏁 Attendance Issue

Update 2/20/19 @ 8:20 AM CT
Attendance is now sending out normally. We've added some internal tracking to know timestamp exactly when each attendance reminder was sent out for each group. We're also going to setup a couple more instances throughout the day where we'll double check that everyone who was supposed to receive the attendance reminder, in fact, received it. If not, it will try re-sending again. Sometimes our email provider (MailChimp's Mandrill) gives us an error when sending out emails and it will stop sending all in that batch after it gets that error, preventing emails from going on. So by adding a few more checks throughout the day, we'll be able to help ensure that emails that didn't get sent before will get sent.

2/19/10 @ 10:00 AM CT 
There's a small percentage of groups that aren't getting their attendance notifications. We're digging in to see why that is.

🏁 Notifications Issue

Update 11/5/18 @ 1:35 PM CT
We got it! We discovered that there was just one affected email notification setting that wasn't working properly. It was the "attendance has been taken" email notification. If you had this notification OFF before our update last week, it was actually being sent out. This primarily affected account admins/staff. If someone was a group leader or member and receiving these notifications, it's because they are also a staff/admin on the account. We've fixed the bug and anyone who had the "attendance has been taken" notification toggled to OFF will no longer receive these notification emails with our fix. We're so sorry for flooding some inboxes with these notifications over the weekend.

Lastly, we fixed the issue with the My Notifications page not saving changes. So you're welcome to head over there and double check your notifications to be sure everything is set the way you currently want it.

Update 11/5/18 @ 11:50 AM CT
We've received reports that email notifications that were toggled to OFF are being sent out and saving some notification settings isn't working. We're currently working on getting this fixed asap.

🏁 Attendance Issue

Update 10/22/18 @ 4:00 PM CT
We found and fixed a bug with our attendance processing. Most groups attendance was submitted that was taken while the bug was in place. However, some groups may have not had their attendance saved/processed. They'll have another chance to do that tomorrow when our attendance not taken yet notices go out.

Update 10/22/18 @ 3:30 PM CT
We've received some reports that attendance isn't saving for some groups. We're currently looking into this.

🏁 Slow Performance

Update 10/5/18 @ 7:30 AM CT
We're back and faster than ever πŸ”₯By now, you should be noticing how fast everything is. We've moved over to Google's Cloud Platform, which has sped things up considerably and now we have unlimited resources to grow and expand because of Google's excellent technology and infrastructure. Thanks again for your patience this week. If there are any issues you run into, please don't hesitate to  contact us.

Update 10/4/18 @ 10:25 AM CT
We're about 85% ready for moving everything to Google Cloud. We're doing lots of testing to ensure everything is fast and functional. If everything goes as planned, we'll switch GroupVitals over to Google tomorrow morning.

Update 10/3/18 @ 10:55 AM CT
We're making progress with moving everything to Google Cloud πŸ‘We've got the basic application up and running and now we're just fine tuning things. I personally just tested it and everything is fast.. faster than it was before πŸŽ‰We've run into a snag with our F1 API that we are trying to fix at the same time as getting everything moved over to Google. If we don't make the move Thursday morning, we'll shoot for Friday morning. Thanks again for your patience!

Update - 10/2/18 @ 1:30 PM CT
We're still on track to move the entire GroupVitals application to Google Cloud this week. The goal is to move it early Thursday morning. Thanks again for your patience with the slower system this week 

Update - 10/1/18 @ 11 AM CT
In order to solve the slowness of the system, we'll be continuing ahead with our plan to move the application to Google's cloud platform. We planned to do within the next few weeks, but we're going to make the move this week. Getting both the database and the application hosted with Google in the same environment will increase the speed and allow us to expand our resources as needed. Thanks for your patience and we'll keep you updated on this page!

Update - 9/28/18 @ 8 AM CT
We've moved our database to Google Cloud early this morning. We're noticing a slower performance in loading pages and saving changes and we're currently working on improving that. We'll update this page when we've got things loading faster.

🏁 Database Move

Update - 9/29/18 @ 8:00 AM CT
The database has been successfully moved over to Google Cloud πŸ™Œ

Update - 9/28/18 @ 12:00 AM CT 
We've started the process of moving our database from a private server to Google Cloud in order for us to better support future growth and better performance. This is the first phase of moving our entire system to Google's Cloud over the next month.