[Resolved] BHRIC Virtual Desktop Infrastructure downtime

The Biomedical Health Research Informatics Core experienced an error that prevented virtual desktops being rebooted, started, or migrated. The error occurred between 7:00 – 7:50 am August 22 and appears to have been resolved. For more information please contact the Research Computing team.

Posted in Educational Technologies, Network and Telecom, Systems, Uncategorized

Owl Apps login issue: Google Services “Something went wrong”

Google status shows the following:

2:50 PM
Description:

We are investigating a potential issue with signing into Google services.

We will provide more information by Monday, 2019-08-19 12:45 US/Pacific.

How to diagnose:

Users may have issues signing into Google services. They may receive the error “Something went wrong. Sorry, something went wrong there. Try again.”

Workaround:

There’s no workaround available at this time, but we’ll let you know if we learn of one.

Posted in Educational Technologies, Email, Instructional Technologies, Systems, Uncategorized

[RESOLVED] Starfish – Down

Starfish is currently down and we are working with Hobsons to resolve the issue.

Posted in Uncategorized

OIT Systems Maintenance, Thurs. July 25 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 07/25/2019 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

*************************************************************************************
Shutting down legacy server (Blackboard)
Description of change: OIT will be shutting down the Blackboard webservers​ that are no longer in use.
Downtime window start: 10:00 AM
Downtime window end: 11:00 AM
Affected services: Blackboard​
Expected End-user impact: During this time, no end-user impact is expected.

*************************************************************************************
Canvas subaccount fix
Description of change: OIT will apply a fix for subaccount query for import into canvas
Downtime window start: 12:00 AM
Downtime window end: 12:00 AM
Affected services: ​Canvas import job
Expected End-user impact: During this time, no end-user impact is expected.

*************************************************************************************
REDcap upgrade
Description of change: OIT will be upgrading the REDCap data collection tool to the latest long term supported release.
Downtime window start: 5:00 AM
Downtime window end: 6:00 AM
Affected services: REDCap​
Expected End-user impact: During this time, REDCap will be unavailable. ​

*************************************************************************************

Posted in Banner and Admin Systems, Instructional Technologies, Systems, Uncategorized

BHRIC Health System Experiencing Issues

Following this weekend’s network updates, the BHRIC Health System is experiencing connectivity issues. Users may have difficulty logging in to VDIs and Servers. The team is working diligently to restore services to normal operation. We will update as progress continues.

Posted in Uncategorized

OIT Systems Maintenance, Thurs. July 18 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 07/18/2019 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

******************************************************************************************************************

Mail Server Configuration Change
Description of change: OIT will be making a change to how email recipients are validated.
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 3:00 AM
Affected services: Inbound email, smtp.fau.edu
Affected systems: boc22fortimail01.fau.edu, nwfortimail01.fau.edu
Expected End-user impact: none

Chessboard Certificate Update
Description of change: Update the SSL certificate on chessboard the User Services management tool. ​
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: Password reset tools for user services
Affected systems: chessboard.fau.edu
Expected End-user impact: none

GitLab Certificate Update and Config Changes
Description of change: OIT will be migrating the mail servers that handle smtp.fau.edu to new servers.
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: gitlab​
Affected systems: gitlab.hpc.fau.edu
Expected End-user impact: Gitlab will be turned off an back on again. ​Users who have used the scheduler may see their jobs run 45 minutes earlier than expected. We are also changing the default time zone from UTC to America/New_York, this affects times displayed in gitlab UI.​

BHRIC certificate updates
Description of change: All SSL certificates used on the BHRIC including, Redcap, VMware, and others will be updated to a new Digicert or LetsEncrypt certificate.
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: BHRIC, VMware Horizon​, Redcap, VMware Virtual Center
​Affected systems: redcap1.fau.edu (letsencrypt), redcap2.fau.edu (letsencrypt), redcapstore.fau.edu (letsencrypt), vhealth.fau.edu (digicert), vhealthon.fau.edu (digicert), boc22-h-hv1.health.ad.fau.edu (digicert), boc22-h-vc1.health.ad.fau.edu (digicert)
Expected End-user impact: Users will need to reconnect to their services when they return to their desks should they be connected to VDI over the evening.

BHRIC Servers upgrade from Windows 2012R2 to Windows 2019
Description of change: We have series of Windows servers on BOCA RATON and JUPITER cloud platforms running Windows 2012 R2. These servers will be upgraded to Windows 2019​. Several servers are upgraded successfully. We would like to continue upgrading all servers in 3/4 servers per batch over next few months. ​​
Downtime window start 7/11/2019 3:00 AM
Downtime window end 7/11/2019 6:00 AM
Affected services: Various services on following servers in health domain: boc22-s-adc1, Boc22-s-ps.med, research-mssqlX, boc22-s-sql
Affected systems: boc22-s-adc1, Boc22-s-ps.med, research-mssqlX, boc22-s-sql
Expected End-user impact: NONE is expected​​​

Touchet Maintenance
Description of change: Migrate Touchnet Payment Transaction web service from Oracle Application Server to Tomcat in order to support Oracle database 12c.
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: Touchnet
Affected systems: ctcache1, sctcache2, boc22banxe03, boc22banxe04
Expected End-user impact: Touchnet will be unavailable.

Banner Database Maintenance
Description of change: The production Banner database will be upgraded from Oracle 11gR2 to 12cR2.​
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: Banner production database and any applications that depend on it. Appworx, Harvest, Workflow, Banner Admin Pages, Banner self-service, Touchnet
Affected systems: bannerprd
Expected End-user impact: Affected services will be down during the time of the maintenance.

Secure file server data migration
Description of change: OIT will be moving the data internally on a ​secure file server
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: all shares on andisec
Affected systems: andisec
Expected End-user impact: all shares on andisec will be unavailable during the window

Mail Server Upgrade
Description of change: OIT will be replacing current mail gateways with new appliances.
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 4:00 AM
Affected services: Inbound mail processing for FAU (Office 365 and Gmail)
Affected systems: New Units:boc22fortimail01.fau.edu, nwfortimail01.fau.edu Old Units: bertx.fau.edu, berty.fau.edu, boc22esa01.fau.edu, nwesa01.fau.edu
Expected End-user impact: Ideally none, however there may be some unintended impact if a configuration is missing/incorrect.

BHRIC Servers upgrade from Windows 2012R2 to Windows 2019
Description of change: We have series of Windows servers on BOCA RATON and JUPITER cloud platforms running Windows 2012 R2. These servers will be upgraded to Windows 2019​. Several servers are upgraded successfully. We would like to continue upgrading all servers in 3/4 servers per batch over next few months. ​​
Downtime window start 7/18/2019 3:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: Various services on following servers in health domain: boc22-s-adc1, Boc22-s-ps.med, research-mssqlX, boc22-s-sql
Affected systems: boc22-s-adc1, Boc22-s-ps.med, research-mssqlX, boc22-s-sql
Expected End-user impact: NONE is expected​​​

Account Self Service Updates
Description of change: Installing patches and updates to backend services. Adding support for Workday Report to Active Directory Group synchronization​.
Downtime window start 7/18/2019 5:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: Accounts Self Service – https://accounts.fau.edu, user manager
Affected systems: um.fau.edu
Expected End-user impact: none, requires a rolling restart of services but should not cause any disruption in services.

Updates to HIPAA Onboarding
Description of change: We are making adjustments to correct an issue related to the on boarding process of HIPAA employees. These changes should not effect existing HIPAA users.
Downtime window start 7/18/2019 5:00 AM
Downtime window end 7/18/2019 6:00 AM
Affected services: ​Database
Affected systems: OIM
Expected End-user impact: This change should correct issues related to passwords synchorzing to newly created HIPAA AD Accounts.

Server Maintenance
Description of change: We will be changing the current www.fau.edu webservers with new webservers that are able to do shibboleth.​
Downtime window start 7/18/2019 6:00 AM
Downtime window end 7/18/2019 7:00 AM
Affected services: www.fau.edu website
Affected systems: www.fau.edu webservers​
Expected End-user impact: No end user impact expted as these webservers are behind bigip.

AutoAdvisor Maintenance
Description of change: Bug fix to the AutoAdvisor integration that exports to StarFish to include a new population for the General Advisor role.​
Downtime window start 7/18/2019 6:45 AM
Downtime window end 7/18/2019 7:00 AM
Affected services: None
Affected systems: ​AutoAdvisor
Expected End-user impact: ​None​

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems, Uncategorized

OIT Systems Maintenance, Thurs. July 10 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 07/10/2019 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

*****************************************************************************************

Server maintenance
Description of change: Certificates for various secure web sites managed by OIT are being renewed
Downtime window start 7/11/2019 3:00 AM
Downtime window end 7/19/2019 7:00 AM
Affected services: Almost all services that rely on our wildcard certificate.
Affected systems: bigip and any other system using the wildcard fau certificate​
Expected End-user impact: varies. some to none depending on the sensitivity of the service on accepting a new certificate.

********************************************************************************************
Secure file server data migration
Description of change: OIT will be moving the data internally on a ​secure file server
Downtime window start 7/11/2019 3:00 AM
Downtime window end 7/11/2019 6:00 AM
Affected services: all shares on andisec
Affected systems: andisec
Expected End-user impact: all shares on andisec will be unavailable during the window

*****************************************************************************************
Mail Server Upgrade
Description of change: OIT will be migrating the mail servers that handle smtp.fau.edu to new servers.
Downtime window start 7/11/2019 3:00 AM
Downtime window end 7/11/2019 4:00 AM
Affected services: Mail relay for smtp.fau.edu
Affected systems: boc22esa02.fau.edu, nwesa02.fau.edu
Expected End-user impact: Ideally none, however depending on how dns is cached for each client and how the smtp server settings were configured you may see bouncebacks/failed to connect messages when dns is updated.

********************************************************************************************
Windows configuration and deployment system upgrade
Description of change: The system providing configuration, deployment and updates to Windows clients will be upgraded to the latest version
Downtime window start 7/11/2019 3:00 AM
Downtime window end 7/11/2019 7:00 AM
Affected services: OS deployments, app installs, windows updates from SCCM​
​Affected systems: boc22sccm
Expected End-user impact: All OS deployments, app installs, windows updates from sccm will be unavailable during the window

**********************************************************************************************
BHRIC Servers upgrade from Windows 2012R2 to Windows 2019
Description of change: We have series of Windows servers on BOCA RATON and JUPITER cloud platforms running Windows 2012 R2. These servers will be upgraded to Windows 2019​. Several servers are upgraded successfully. We would like to continue upgrading all servers in 3/4 servers per batch over next few months. ​​
Downtime window start 7/11/2019 3:00 AM
Downtime window end 7/11/2019 6:00 AM
Affected services: Various services on following servers in health domain: boc22-s-adc1, Boc22-s-ps.med, research-mssqlX, boc22-s-sql
Affected systems: boc22-s-adc1, Boc22-s-ps.med, research-mssqlX, boc22-s-sql
Expected End-user impact: NONE is expected​​​

**********************************************************************************************
Banner Self-Service Maintenance
Description of change: Change to fix flaws associated with Banner Registration (Version 9.12) that causes inaccurate information to be displayed to the students. This fix includes updates to the Registration search feature to fix defects on what courses are rendered and fix inaccurate descriptions/sorting of other data points.
Downtime window start 7/11/2019 4:00 AM
Downtime window end 7/11/2019 6:00 AM
Affected services: Banner Self-Service
Affected systems: Banner XE
Expected End-user impact: There are no impact on user services expected

**************************************************************************************
Updates Accounts Self Service
Description of change: Upgrading and patching account self service website.
Downtime window start 7/11/2019 5:00 AM
Downtime window end 7/11/2019 6:00 AM
Affected services: accounts.fau.edu
Affected systems: boc22oimwebp1, boc22oimwebp2, nwrdcoimwebp1, nwrdcoimwebp2
Expected End-user impact: Users may see a delay in password resets during the time the systems are being patched and rolled out.

*************************************************************************************

Server Maintenance
Description of change: We will be changing two northwest www.fau.edu webservers with new ones that are able to do shibboleth.
Downtime window start 7/11/2019 6:00 AM
Downtime window end 7/11/2019 7:00 AM
Affected services: None as these are the northwest www.fau.edu webservers and are also behind BigIP
Affected systems: Northwest www.fau.edu webservers​
Expected End-user impact: No end user impact expected

Posted in Banner and Admin Systems, Educational Technologies, Network and Telecom, Systems, Uncategorized

Unable to load images in VIP document management system

We are currently experiencing issues with the content explorer component of the VIP document management system.  As such, users are unable to load images.  Scanning and input should still be functional.  Working with vendor for a solution.

Posted in Banner and Admin Systems, Systems, Uncategorized

[RESOLVED] Change of Grade workflow unavailable

The Change of Grade workflow process is now available.

Change of Grade workflow process is currently unavailable at this time. We are currently investigating the issue and working to resolve the issue as soon as possible and we appreciate your patience during this time..  We do not currently have an estimated time of resolution (ETR).

OIT

Posted in Uncategorized

[RESOLVED] TouchNet Systems Maintenance, Wed. June 12th from 3:00AM to 7:00AM

The TouchNet payment processing system will be undergoing maintenance -Starting from 3:00am to 7:00am (EST) Wednesday, 06/12/2019

This affects student payment services (Bill+Payment), as well as all Marketplace Stores, uPay sites and TouchNet ready partner payment services.

We expect all systems to be back online no later than 7:00 am (EST) that morning. When the system is back up and ready to go, we will notify you.

You may wish to place warnings on your website, or take your store offline. If this scheduled downtime creates a conflict for your operations, please feel free to contact us at FIS@fau.edu for further discussion.

Posted in Uncategorized