Resolved: HyperV services down – numerous VMs are down

Services down:  idm gateway, sharepoint, dns resolutions, talapp, ldap and others that are located on the cluster.   Systems is troubleshooting.

Posted in Network and Telecom

OIT Systems Maintenance, Thurs. Dec 19 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 12/19/2013 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

When: 3:00AM – 4:00AM
Affected services: WordPress
Affected systems: WordPress
Description of maintenance: We are doing updates to enhance wordpress functionality
User impact: No end user impact is expected

When: Thursday, December 19, 2013: 3:00-5:00 AM, (Friday December 20, 2013 5:00AM-7:00AM if necessary).
Affected services: Blackboard
Description: Blackboard has released an update to several Building Blocks including SafeAssign, Collaborate, Blackboard Mobile, Cloud Profiles, Retention Center, and Content Editor. Additionally, publishers Pearson, McGraw Hill and Cengage have released updated Building Blocks. We will also be updating the Advanced User Search Building Block, and installing two new Building Blocks, My Surveys and Quickly. We will get as many of these done as we can during the time frame on Thursday. Any remaining Building Blocks will be updated on Friday morning. We do not expect any downtime.

When: 5:30AM – 6:00AM
Affected services: Network
Affected systems: Network
Description of maintenance: The goal of this change is to enable new network functionality for FAU. OIT will perform required changes to the network infrastructure.
User impact: No end user impact is expected.

When: 5:00AM – 6:00AM
Affected services: Network
Affected systems: Cisco Nexus core router
Description of maintenance: The goal of this change is to upgrade the software on the Cisco Nexus core router.
User impact: This change will result in a brief network outage as the upgrade is completed. All devices on the FAU network should reconnect after the upgrade and reboot has been completed.

Posted in Network and Telecom

OIT Systems Maintenance, Thurs. Dec 12 from 3:00AM to 7:00AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 12/12/2013 from 3:00 AM to 7:00 AM during OIT maintenance window. In addition, the following systems will be specifically impacted:

When: 3:00AM – 7:00AM
Affected services: Banner Production
Affected systems: Banner Production
Description of maintenance: The following modules will be installed to the specified version in production FAUP: Finance Self Service 8.6.0.2 patch and Finance 8.9.0 upgrade
User impact: There will be no end user impact.

When:5:00AM – 5:30AM
Affected services: Websites or Apps relying on databases on mysql01
Affected systems: mysql01
Description of maintenance: We are making changes to simplify the data backup and recovery process for the mysql database server.
User impact: Websites or apps relying on databases located on mysql01 will be unavailable during the maintenance window.

When: 8:00AM – 2:00PM
Affected services: Echo360
Affected systems:
Echo360
Description of maintenance: The Echo class video system is being updated to improve response client times with the system. Along with the software update to 5.3 SP2, we will also be separating the web services component of the Echo360 system to a new virtualized environment. Currently the Echo system management server is also hosting the web services for this system. Separating out this service will allow the system to better handle client viewing traffic load.
User impact: End-user impact will be minimized at this time, as regular term classes will no longer access content stored in the Echo system. However the College of Medicine has ongoing classes that do use the Echo360 system to distribute class video content; they confirmed this date/time is best for their class schedule.
Echo Player streaming class videos as well as audio/video podcast downloads will be available intermittently during this maintenance window.
Also, the Echo management interface at https://echo.fau.edu will be intermittently available during this time. This will affect the publishing of Echo Personal Capture and External Media Ingest videos. Because of this, we recommend postponing the publishing of any Personal Capture or External Media.

Posted in Network and Telecom

OIT Systems Maintenance, Wednesday, Nov 27 from 3:00am to 7:00am

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Wednesday, 11/27/2013 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

When:  3am to 6am.

Affected Services: IDM production, Luminis production, Phonebill app, Report Caster production, IEA Production database IEAP, Legacy Travel prod database FINP.

Affected Systems: boc22ora01, boc22ora03,boc22ora05 , mcoracle, nwrdcora01.

User Impact:  Affected services will be unavailable during the maintenance window.

Reason: Oracle database patch set updates and host patch installations

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems

Emergency Network Maintenance – Wednesday November 26th from 3:00a.m. until 3:30a.m.

Communications Infrastructure will be performing network maintenance from 3:00a.m. until 3:30a.m. on Wednesday November 26th to resolve issues with network slowness. During this time there may be a short network outage lasting approximately 5 minutes. If such an outage occurs, this outage will affect connectivity between FAU and the internet.  This maintenance is expected to temporarily resolve the following issues that have been reported by end users:

Frequent pauses in Echo360 recordings

Poor Voice-over-IP quality (Non-FAU VoIP providers such as Skype)

Slow / Laggy performance of Remote Desktop sessions for VPN users

Slow / Laggy performance in online games from on-campus housing

Posted in Banner and Admin Systems, Educational Technologies, Network and Telecom, Systems

Resolved: Echo360 Service Emergency Restart

RESOLVED:  Echo360 will be intermittently unavailable from 3:30-4:00 pm.  The Echo360 service needs to be restarted to free a blocked thread that is preventing content from being ingested.  Any user viewing an Echo recording will have their service interrupted while the item restarts.

 

The service has been restarted and captures are now being  routed to the processing queue as designed.

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems

OIT service interruption [update]

OIT has identified a storage problem which was the cause for the service outage. The problem has been corrected and services are returning to their normal state. We thank you for your patience.

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems

OIT service interruption

OIT has experienced a problem in its data center which are affecting multiple services, some of which are:

– DNS
– Blackboard
– Sharepoint
– Internal websites

We are investigating the problem and will post another update at 7:30am. We appreciate your patience.

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems

OIT Systems Maintenance, Thurs. Nov 21 from 3:00 – 7:00 AM

Office of Information Technology will be performing routine systems maintenance which includes but is not limited to rebooting Windows servers as deemed necessary, installing critical patches and other health checks on Thursday, 11/21/2013 from 3:00 AM to 7:00 AM during OIT maintenance window.  In addition, the following systems will be specifically impacted:

When: 6:00 – 7:00AM
Affected services: smtp.fau.edu
Affected systems: smtp.fau.edu
Description of maintenance: We are restructuring and consolidating mail servers to simply the email flow architecture and to enforce spam and virus scanning.
User impact: End-users in this change will refer more to users running various mail servers or services on campus: 1. Possible rate limiting until we add server to exception list. 2. Email originating from exception list servers (e.g. blackboard, ibiweb) will not be virus or spam scanned. 3. Possible firewall blocks until security adds new rules for server.

When: 5:00 – 6:00AM
Affected services: swise.fau.edu
Affected systems: swise.fau.edu
Description of maintenance: We are freeing the swise.fau.edu server from its restrictive, physical shell to allow it to become a virtual entity.
User impact: https URLs and apps served by swise.fau.edu will be unavailable.  These include: Clearwire, Electronic Thesis, Scholarship Management System, Strategies & Tips for Academic Recovery, Writing Across Curriculum, and True Blue.

When: 5:00 – 7:00AM
Affected Services: ODS production environment. Webfocus won’t be able to connect to the ODS database.
Affected Systems: odsprod
User Impact:  Affected services will be unavailable during the maintenance window.
Reason: Oracle database ODSP needs to be patched for ODS 8.4.1

When: 3:00 – 7:00AM
Affected services: Banner FAUP: Accounts Receivable upgrade 8.4.4, Accounts Receivable upgrade 8.4.5, Student Self Service 8.5.4.3 & 8.5.4.4 patches
Affected systems: Banner FAUP
Description of maintenance: The following updates will be installed in Banner production: Accounts Receivable upgrade 8.4.4, Accounts Receivable upgrade 8.4.5, Student Self Service 8.5.4.3 & 8.5.4.4 patches
User impact: FAUP may be unavailable during this time

When: 3:00 – 6:00AM
Affected Services: Banner production, Blackboard production
Affected Systems: bannerprd, bannerprddr, bbloracle, bbloracledr.
User Impact:  Affected services will be unavailable during the maintenance window.
Reason: Oracle database patch set updates and host patch installations

When: 4:00AM – 6:00AM
Affected services: secure.fau.edu, secure2.fau.edu, ulrweb.fau.edu, fauftp1
Affected systems: FAUWeb1, FAUWeb6
Description of maintenance: OIT will be migrating the secure.fau.edu, secure2.fau.edu, and ulrweb.fau.edu websites to a new server. Additionally, fauftp1 will move as well.
User impact: Secure.fau.edu, secure2.fau.edu, ulrweb.fau.edu and fauftp1.fau.edu will be intermittently inaccessible during the window. These sites also affect secure andi, student and home pages; plus the mealplans, oig, orientation, dotnetdev, and changeGappsPassword sites.

When: 4:00 – 6:00AM
Affected services: Talisma
Affected systems: Talisma
Description of maintenance: We will be applying the newest hot fix to talisma per the request of admissions. Talisma will be unavailable from 4-6 am. This new patch also requires the talisma client to be patched.  Admissions will contact their users to let them know about the downtime and the client change.
User impact: Talisma will be unavailable from 4-6 am.

 

Posted in Banner and Admin Systems, Educational Technologies, Email, Network and Telecom, Systems

Network: Parliament Hall Building 102 partial network outage

We are experiencing a partial network outage in the Parliament Hall Building #102 that is affecting the second floor.

Thank you

Posted in Network and Telecom