[RESOLVED] Slowness with some services

** UPDATE **

All services are operating normally at this time.

*************************************

We are currently experiencing slowness with some services, such as Workday, email, Banner INB, Self-service, and possibly some others. We are currently investigating the issue and will post an update once we know more.

We apologize for the inconvenience.

OIT

Posted in Banner and Admin Systems, Educational Technologies, Email, Google App Status Feed, Instructional Technologies, Network and Telecom, Systems, Uncategorized, Workday

[RESOLVED]Password change/reset delays

Status: 6/21/17 6:45PM

The password change/reset delays have been resolved and are back to functioning as normal.

We appreciate your patience and support.

OIT

————————————————————

Status: 6/21/17 5:30PM

Please be advised that the issue with password change delays has been resolved, however, the system is currently processing the password changes and should normalize in no more than a couple of hours. If you change your password during this time please be aware that it may take some time to propagate to all systems.

We appreciate your patience.

OIT

 

_________________________________________________________________________________

Password changes/reset is currently experiencing extended delays. We recommend holding off changing your password until further notice.

We are working with due diligence to resolve this issue.

We appreciate your patience during this time.

OIT

Posted in Banner and Admin Systems, Educational Technologies, Email, Google App Status Feed, Instructional Technologies, Network and Telecom, Systems, Uncategorized, Workday

Complete: Single Sign On (SSO) maintenance

The Single Sign On (SSO) maintenance has been completed successfully and Single Sign On is now online.

Posted in Banner and Admin Systems, Educational Technologies, Email, Instructional Technologies, Network and Telecom, Systems, Workday

[UPDATED NOTICE] OIT Systems Maintenance, Thurs. June 8 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 06/08/2017 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

Single Sign On (SSO) maintenance
Description of change: We will be updating the Single Sign On​ logout screen to include the following changes: ​
1. Remove Blackboard and Exchange WebMail links
2. Add Canvas link
Downtime window start: 3:00 AM
Downtime window end: 7:00 AM
Affected services:  All Single Sign on systems may go offline including, Canvas, Blackboard, Workday, Google Applications, Office 365, and many more.
Affected systems: shibboleth1, shibboleth2, shibboleth3
Expected End-user impact: During the maintenance window, access to applications utilizing FAU NetID logins may be intermittently unavailable including Canvas, Blackboard, Workday, Google Applications, and Office 365

*************************************************************************************

Server maintenance
Description of change: OIT will be upgrading the VM version of several systems as well as expanding the processor capabilities for some VMs​​​​
Downtime window start: 3:00 AM
Downtime window end: 4:00 AM
Affected services: Labs print server, Appadmin, Canvas import tool, Networking DHCP server, PaperCut, Talisma, College of Medicine admissions website, AFTSS server, University Police – wireless lock control, High Performance Computing Primary DNS and Web Server, IDM, eGrades and BbGoogle​, Starfish, Undergrad app, Middleware Identity Management
Affected systems: appadmin, as5, boc22canvascli, boc22cm102srv10, boc22cm102vrt101mrtg, boc22ora09, boc22pcapp01, boc22pclyprt01, boc22pcoffice01, boc22talweb2, comvmpsql1, comvmweb1, comvmwebdev1, controller, dsr2, dsr3, dsr5, duo, hpc, idm2, idm4p-oel6, madcat1p, pinky, shibboleth2, starfish, ugapp2, wildflymc1, wildflymc2, yakko​
Expected End-user impact: During the maintenance window, access to services including eGrades, Talisma, Lab Printing, Undergrad admission applications, and College of Medicine admission applications will be intermittently unavailable.

*************************************************************************************

Network Registration maintenance
Description of change: The network registration system will be receiving an updated theme and be switched to a more reliable authentication mechanism. ​
Downtime window start: 3:00 AM
Downtime window end: 6:00 AM
Affected services: All services running on Talon.fau.edu including Chessboard, PrivateEye, Google Password Reset, Password Testing Tool, and others​ will be affected by this update.
Affected systems: talonx1, talonx2
Expected End-user impact: During the maintenance window, access to network registration and other services at the talon.fau.edu domain name may be intermittently unavailable.

*************************************************************************************

Boca Desktop Software Changes
Description of change:  ​Alertus Desktop software will be installed on user computers.
Downtime window start: 3:30 AM
Downtime window end: 4:00 AM
Affected services: ​N/A
Affected systems:  User PCs​ within FAU\BOC OU
Expected End-user impact: No end-user impact regarding service access is expected. New software will be pushed to computers on Boca campus and part of the FAU Active Directory Domain. As a result, there will be a new icon in the system tray. This icon will be for the new desktop alert system.

*************************************************************************************

Oracle database maintenance
Description of change: Applying the Oracle upgrade to FAUDW db on host odsprod
Downtime window start: 3:00 AM
Downtime window end: 5:00 AM
Affected services: FAUDW production environment. Webfocus won’t be able to connect to the FAUDW database.​
Affected systems: Odsprod​
Expected End-user impact:  During the maintenance window, access to Webfocus will be intermittent

*************************************************************************************

Oracle quarterly DB patches (odsprod)
Description of change:  Applying Oracle PSU to DB’s on host odsprod​
Downtime window start: 5:00 AM
Downtime window end: 6:00 AM
Affected services: ODS and FAUDW production environment. Webfocus won’t be able to connect to the ODS or FAUDW databases​
Affected systems: Odsprod
Expected End-user impact: Users will be unable to login to ODS or FAUDW during the period the database is offline.  This should be no longer than 1 hour.​​

*************************************************************************************

Verasmart upgrade
Description of change: Upgrading Verasmart phone billing server to version 11.3​
Downtime window start: 11:00 PM
Downtime window end: 12:00 AM
Affected services: Phone billing​
Affected systems: Boc22comm1​
Expected End-user impact:  No end-user impact is expected.

*************************************************************************************

Identity Management Maintenance
Description of change: ​OIT will be performing maintenance on our Identity Management System.
Downtime window start: 5:00 AM
Downtime window end: 6:00 AM
Affected services: OIM, OUD, MyFAU, Waveset
Affected systems: boc22oimp1, boc22oimp2​
Expected End-user impact: Changes are being performed on services that Google Applications, MyFAU and password changes rely on. No end-user impact is expected.

*************************************************************************************

 

Server maintenance
Description of change: Maintenance will be performed on a redundant DNS server. DNS servers allow a user to access websites and services by using an easy-to-remember name
Downtime window start: 5:00 AM
Downtime window end: 5:30 AM
Affected services:  ​External facing DNS​
Affected systems: ​ns1
Expected End-user impact: No end-user impact is expected.

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems

[RESOLVED] Single Sign On issues

We are currently experiencing issues with single-sign on. The user will be presented with an error message when attempting to sign in to services such as Workday, Canvas, Helpdesk, Blackboard, and any other application that is utilizing single-sign on. Our team is working on restoring services as soon as possible.

We apologize for the inconvenience this may cause.

OIT

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems, Workday

Google Docs phishing Scam

Google is currently investigating a phishing email that is appearing as Google Docs. Please do not click on the link for any unknown shared google doc requests. If you have clicked the link, please make sure to change your Owl Apps and FAU password as soon as possible.

For further assistance resetting your password or any questions please contact the OIT help desk at 561-297-3999.

OIT

Posted in Educational Technologies, Email, Google App Status Feed

[RESOLVED] – Single-Sign On issues

**** UPDATE 8:46AM *****

The issue with single-sign on has been resolved and all services have been restored at this time. You may need to close all open browser windows and open a new session before attempting to login.

We thank you for your patience during this time.

OIT

 

*************************************************************************************************************

We are currently experiencing issues with single-sign on. The user will be presented with an error message when attempting to sign in to services such as Workday, Canvas, Helpdesk, Blackboard, and any other application that is utilizing single-sign on. Our team is working on restoring services as soon as possible.

We apologize for the inconvenience this may cause.

OIT

Posted in Banner and Admin Systems, Educational Technologies, Email, Google App Status Feed, Instructional Technologies, Network and Telecom, Systems, Workday

OIT Systems Maintenance, Thurs. April 13 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 4/13/2017 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

***************************************************
Migration of servers ds0p/ds1p from old BigIP to the new BigIP
When:  5:00AM – 6:00AM
Affected services:  We expect that there might be ldap disruptions for 1 minute during the move.
Affected systems:  This will likely affect any systems using ds0p and ds1p for ldap lookups.
Description of maintenance: We are continuing the move of servers from the older BigIP load balancer hardware to the new ones.  This move will be for the backend servers that support ldap and changing the IP for ds0p and ds1p.
User impact:  Some services that use ldap may not function during the downtime (1-5 minutes)​

***************************************************
Upgrade to the deployment and Windows client management server (SCCM)
When:  5:00AM – 6:00AM
Affected services:  All OS/application/windows update deplyoments from boc22sccm​​
Affected systems:  boc22sccm​
Description of maintenance: OIT will be upgrading the System Center Configuration Manager server (boc22sccm) to version 1702​. SCCM clients may update to version 1702 over the next week.​​
User impact:  All OS/application/windows update deplyoments from boc22sccm will be unavailable during the upgrade​​

***************************************************
Ironport Upgrade to v10.0.1
When:  4:00AM – 6:00AM
Affected services:  Various email exchanges that cross the ironports, mostly incoming from the outside, but also emails that cross from gmail to office 365, vice versa, list email and possibly others. Internal Office365 to Office365 and internal Gmail to Gmail would not be affected.
Affected systems:   bertx, berty, ironportnw2​
Description of maintenance:  We are upgrading the incoming email spam protection servers to a newer version to fix bugs and gain new features
User impact: None​ significantly as each server will be done separately while the other two servers will pick up the email communication.

***************************************************
Multiple systems down for upgrade
When: 3:00AM – 4:00AM
Affected services:  Undergrad app, Blackboard, SCCM, OARS, FileLocker, Labs servers, Gitlab, Lists, Papercut, SharePoint onprem, Malware bytes​
Affected systems:  ugapp1, bbm1, boc22sccm, boc22oars, filelocker1, ls-snap, ls-osd, gitlab, boc22mbes, boc22spwfe1, boc22spapp1, lists, boc22pcprt01, ls-medmodel, filelocker2
Description of maintenance:  OIT will be upgrading the VM version of several systems as well as expanding the processor capabilities for some VMs​
User impact:  The services listed above may be unavailable for a few minutes during the window

***************************************************
Windows authentication system migration
When:  3:00AM – 7:00AM
Affected services:  AD authentication​​
Affected systems:  davdcfau01, davdcirm01, bocdcfau01
Description of maintenance:  OIT will be migrating an AD domain controller to server 2016​​
User impact:  No end-user impact is expected​​

***************************************************
Encryption method changing on secure fileserver
When:  3:00AM – 7:00AM
Affected services:  ANDISEC
Affected systems: ANDISEC
Description of maintenance:  OIT will be removing the IPSEC requirement for connecting to andisec and enforcing SMB encryption​
User impact:  Fileshares on andisec will be unavailable for about 10 minutes during the window​

***************************************************
Upgrade vWorkspace for BHRIC
When:  3:00AM – 6:00AM
Affected services:  vWorkspace for the BHRIC VDI environment​
Affected systems: boc22-s-vws01, jup17-s-vws01, boc22-s-sa, jup17-s-sa, boc22-s-vws-web, jup17-s-vws-web​, boc22-s-hy1, boc22-s-hy2, boc22-s-hy3, boc22-s-hy3, boc22-s-hy4
Description of maintenance:  ​Upgrading Quest vWorkspace from 8.6.1 to 8.6.3
User impact:  ​End users will not have their deployed applications (SAS, eClinical Works) and desktops. The web gateway will also be down.​

***************************************************
Banner General 8.8.6 – 8.9.0 upgrades and patches
When:  3:00AM – 7:00AM
Affected services:  Banner will be unavailable during installation but that effect should be minimized by installing the upgrades/patches during the maintenance window.
Affected systems:  ​Banner 8 General module
Description of maintenance:  ​These Banner 8 upgrades and patch releases represent the basic infrastructure of the Banner system and are required to promote all of the remaining Banner 8 modules and Banner XE Administrative pages into production
User impact:  None expected

***************************************************
Banner XE Admin Common 9.3.2.4/9.3.3.4 administrative pages
When:  3:00AM – 7:00AM
Affected services:  ​Banner
Affected systems:  Banner
Description of maintenance: ​These two upgrades are required as the initial support of the Banner XE system administrative pages which will be in place of the Banner 8 system forms. Admin Common must be in place before the other pages can be deployed
User impact:  None expected

***************************************************
Banner XE Financial Aid 9.3.1.2 administrative pages
When:  3:00AM – 7:00AM
Affected services:  Banner Financial Aid
Affected systems: Banner Financial Aid
Description of maintenance: This upgrade represents the new front end system for Financial Aid in Banner XE. These pages upgrade the Banner 8 forms currently in production
User impact: None expected

***************************************************
Banner XE General 9.3.2.1 administrative pages
When:  3:00AM – 7:00AM
Affected services:  ​The Banner General module will be unavailable during installation
Affected systems:  ​​The Banner General module will be unavailable during installation
Description of maintenance:  ​This upgrade rperesents the new frontend system for the General module in Banner XE. These pages upgrade the Banner 8 forms currently in production
User impact: ​Minimal user downtime is expected

***************************************************

Posted in Banner and Admin Systems, Email, Instructional Technologies, Systems

OIT Systems Maintenance, Thurs. March 30 from 3:00AM – 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 03/30/2017 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

***************************************************
Windows authentication system migration
When:  3:00AM – 7:00AM
Affected services:  AD authentication​​
Affected systems:  bocdcirm02, jupdcfau01, jupdcirm01
Description of maintenance: OIT will be migrating an AD domain controller to server 2016​
User impact: No end-user impact is expected​

***************************************************
Email changes for Proofpoint email gateway evaluation
When:  5:00AM – 6:00AM
Affected services:  Email
Affected systems:  Ironport email gateways, Gmail, Office365, on-premises exchange​
Description of maintenance: We are evaluating a new email security gateway product and it is necessary to insert the product into the existing flow of email.​
User impact: No end-user impact is expected

***************************************************
Tms reconfiguration on BigIP
When: 5:30PM – 6:00PM
Affected services:  TMS
Affected systems:  tms.fau.edu
Description of maintenance:  We are correcting an issue with the network traffic flow of tms.fau.edu​
User impact: None – no tms sessions are scheduled for the rest of the day

***************************************************
Division of Research SharePoint Site Collection Migration
When: 8:00PM – 9:00PM
Affected services: Division of Research SharePoint Site Collection and any subsites – https://sharepoint.fau.edu/research/
Affected systems: ​On-Premises SharePoint
Description of maintenance:  We will be completing the migration of the Division of Research SharePoint site to Office 365.
On-Prem URL: https://sharepoint.fau.edu/research/
365 URL: https://fau.sharepoint.com/sites/research/
User impact: Users will need to update any bookmarks, saved links, and any calendars that point to https://sharepoint.fau.edu/research/

***************************************************

Posted in Email, Systems

[Resolved] Delivery Status Notification (Failure)

From the period of 10am to 3pm today (3/2), some of you may have erroneously received a Delivery Status Notification (Failure) email for emails being sent to FAU recipients.   The details would have said:

5.1.0 – Unknown address error 550-‘5.7.1 Relaying denied’

Rest assured that your message was properly delivered.  The bounce back message that you received was in error due to improper reporting.   We apologize for the inconvenience.

Posted in Email, Systems