Enron Mail

From:40enron@enron.com
To:
Subject:Weekend Outage Report for 9-14-01 through 9-17-01
Cc:
Bcc:
Date:Thu, 13 Sep 2001 14:45:52 -0700 (PDT)


------------------------------------------------------------------------------------------------------
W E E K E N D S Y S T E M S A V A I L A B I L I T Y

F O R

September 14, 2001 5:00pm through September 17, 2001 12:00am
------------------------------------------------------------------------------------------------------

SCHEDULED SYSTEM OUTAGES: NA

ARDMORE DATA CENTER - FACILITY OPERATIONS: No Scheduled Outages.

AZURIX: No Scheduled Outages.

EB34 DATA CENTER - FACILITY OPERATIONS: No Scheduled Outages.

EDI SERVER:
Impact: CORP
Time: Sat 9/15/2001 at 11:00:00 PM CT thru Sun 9/16/2001 at 3:00:00 AM CT
Sat 9/15/2001 at 9:00:00 PM PT thru Sun 9/16/2001 at 1:00:00 AM PT
Sun 9/16/2001 at 5:00:00 AM London thru Sun 9/16/2001 at 9:00:00 AM London
Outage: Upgrade EDINET
Environments Impacted: Corp
Purpose: Improve reliability, performance, and increase information output.
Backout: 1. Switch IP DNS Entries for the Corp Domain for edinet.enron.com and ecnet.enron.com back to the original state
2. Restore database tables for edi_hub systems.
3. Switch IP DNS Entries for the Corp Domain for edinet.enron.com and ecnet.enron.com.
4. Save old code and script changes and install new code and configure new code from edi_qa.
Contact(s): John Kratzer 713-345-4642 713-318-4746
Jack Metkus 713-853-0362 713-288-0743

EES:
Impact: EES
Time: Fri 9/14/2001 at 5:30:00 PM CT thru Fri 9/14/2001 at 6:45:00 PM CT
Fri 9/14/2001 at 3:30:00 PM PT thru Fri 9/14/2001 at 4:45:00 PM PT
Fri 9/14/2001 at 11:30:00 PM London thru Sat 9/15/2001 at 12:45:00 AM London
Outage: Minor PCCS Production Changes
Environments Impacted: EES
Purpose: Planned, requested enhancements to system
Backout: Restore from backup
Contact(s): Burt Bailey 713 853-9164 888-886-7432
Suchitra Narra 713-345-3049

Impact: EES
Time: Fri 9/14/2001 at 5:00:00 PM CT thru Fri 9/14/2001 at 5:30:00 PM CT
Fri 9/14/2001 at 3:00:00 PM PT thru Fri 9/14/2001 at 3:30:00 PM PT
Fri 9/14/2001 at 11:00:00 PM London thru Fri 9/14/2001 at 11:30:00 PM London
Outage: Presentation Library
Environments Impacted: EES Intranet
Purpose: Business needs these changes ASAP.
Backout: restore from backup
Contact(s): Dharmendra Garg 713.345.3108 866.471.9801

ENRON CENTER SOUTH DATA CENTER - FACILITY OPERATIONS: No Scheduled Outages

ENRON NORTH AMERICAN LANS:
Impact: NY Metals
Time: Fri 9/14/2001 at 7:30:00 PM CT thru Fri 9/14/2001 at 8:00:00 PM CT
Fri 9/14/2001 at 5:30:00 PM PT thru Fri 9/14/2001 at 6:00:00 PM PT
Sat 9/15/2001 at 1:30:00 AM London thru Sat 9/15/2001 at 2:00:00 AM London
Outage: NY Metals - CAT OS Upgrade
Environments Impacted: nY Metals
Purpose: Necessary to implement a Security Patch
Backout: Revert to the previous version of Code
Contact(s): Pete Link 713-345-8102

Impact: ECT Calgary
Time: Sat 9/15/2001 at 9:00:00 AM CT thru Sat 9/15/2001 at 12:00:00 PM CT
Sat 9/15/2001 at 7:00:00 AM PT thru Sat 9/15/2001 at 10:00:00 AM PT
Sat 9/15/2001 at 3:00:00 PM London thru Sat 9/15/2001 at 6:00:00 PM London
Outage: Calgary Core Upgrade
Environments Impacted: ECT Calgary
Purpose: The 4000's do not have the capacity to support the office.
Backout: Put the 4000's back in production.
Contact(s): Mark Delaro 713-345-9952

FIELD SERVICES: No Scheduled Outages.

INTERNET: No Scheduled Outages.

LOTUS NOTES:
Impact: CORP
Time: Fri 9/14/2001 at 5:15:00 PM CT thru Fri 9/14/2001 at 6:15:00 PM CT
Fri 9/14/2001 at 3:15:00 PM PT thru Fri 9/14/2001 at 4:15:00 PM PT
Fri 9/14/2001 at 11:15:00 PM London thru Sat 9/15/2001 at 12:15:00 AM London
Outage: Production LN Server Migration
Environments Impacted: Corp
Purpose: Server need to be in production server environment
Backout:
Contact(s): Dalak Malik 713-345-8219

MESSAGING:
Impact: Nahou-mspfs01p 02p 03p Nahou-mscnx07p Nahou-msims01p 02p Nahou-msimr01p
Time: Sun 9/16/2001 at 7:00:00 AM CT thru Sun 9/16/2001 at 3:00:00 PM CT
Sun 9/16/2001 at 5:00:00 AM PT thru Sun 9/16/2001 at 1:00:00 PM PT
Sun 9/16/2001 at 1:00:00 PM London thru Sun 9/16/2001 at 9:00:00 PM London
Outage: Remaining E2k server upgrade
Environments Impacted: All Exchange users will be effected
Purpose: Need to upgrade remaining Houston Exchange servers to the same versions
Backout: Restore from backup.
Contact(s): David Lin 713-345-1619

MARKET DATA: No Scheduled Outages.

NT: No Scheduled Outages.

OS/2: No Scheduled Outages.

OTHER SYSTEMS:
Impact: CORP
Time: Fri 9/14/2001 at 5:00:00 PM CT thru Sun 9/16/2001 at 11:00:00 PM CT
Fri 9/14/2001 at 3:00:00 PM PT thru Sun 9/16/2001 at 9:00:00 PM PT
Fri 9/14/2001 at 11:00:00 PM London thru Mon 9/17/2001 at 5:00:00 AM London
Outage: Large-Scale Data Migration from Nova (Unix Devel)
Environments Impacted: Unix
Purpose: solar and nova (old auspex servers) are being decommissioned
Backout: Unmount clients. Copy back saved maps. Re-enable NFS services on Nova. Scan NetApps for changes and copy them over.
Contact(s): Mike Croucher
Adam DeRidder (713) 853-1428
Carlos Uribe (713) 345-8179
Bob McCauliffe

Impact: SAP
Time: Fri 9/14/2001 at 5:00:00 PM CT thru Mon 9/17/2001 at 2:00:00 AM CT
Fri 9/14/2001 at 3:00:00 PM PT thru Mon 9/17/2001 at 12:00:00 AM PT
Fri 9/14/2001 at 11:00:00 PM London thru Mon 9/17/2001 at 8:00:00 AM London
Outage: SAP 9-1 Release
Environments Impacted: SAP
Purpose: Quarterly release.
Backout: Restore from prior backup.
Contact(s): Gary Sentiff 713-345-5125

Impact: CORP
Time: Fri 9/14/2001 at 8:00:00 PM CT thru Sun 9/16/2001 at 4:00:00 PM CT
Fri 9/14/2001 at 6:00:00 PM PT thru Sun 9/16/2001 at 2:00:00 PM PT
Sat 9/15/2001 at 2:00:00 AM London thru Sun 9/16/2001 at 10:00:00 PM London
Outage: TAGG/ERMS Oracle Database V8.1.7.2 Migration
Environments Impacted: ENA
Purpose: Keep TAGG/ERMS database at the version level supported by Oracle. Make use of the latest proven technology.
Backout: Restore the database from the backup taken before the upgrade starts.
Contact(s): Alex Hsieh 713-853-6561 713-288-1779

Impact: CORP
Time: Sat 9/15/2001 at 10:00:00 AM CT thru Sat 9/15/2001 at 5:00:00 PM CT
Sat 9/15/2001 at 8:00:00 AM PT thru Sat 9/15/2001 at 3:00:00 PM PT
Sat 9/15/2001 at 4:00:00 PM London thru Sat 9/15/2001 at 11:00:00 PM London
Outage: Maintenance of Unix server 'Frissure'.
Environments Impacted: Corp
Purpose: This addresses a none CPU bug and provides additional redundancy.
Backout:
Contact(s): Wilford Stevens 713-853-7564
Rajiv Varma 713-345-4955

Impact: CORP
Time: Sat 9/15/2001 at 12:00:00 PM CT thru Sun 9/16/2001 at 12:00:00 AM CT
Sat 9/15/2001 at 10:00:00 AM PT thru Sat 9/15/2001 at 10:00:00 PM PT
Sat 9/15/2001 at 6:00:00 PM London thru Sun 9/16/2001 at 6:00:00 AM London
Outage: Maintenance of Unix ClearCase Client machines
Environments Impacted: All Development Groups
Purpose: To insure that Client install is update with host.
Backout: Uninstall patches.
Contact(s): Troy Beyer 713.345.8316 713.406.1782

Impact: CORP
Time: Sat 9/15/2001 at 6:00:00 PM CT thru Sun 9/16/2001 at 12:00:00 AM CT
Sat 9/15/2001 at 4:00:00 PM PT thru Sat 9/15/2001 at 10:00:00 PM PT
Sun 9/16/2001 at 12:00:00 AM London thru Sun 9/16/2001 at 6:00:00 AM London
Outage: O/S upgrade on Unix server 'Pulsar'.
Environments Impacted: Corp
Purpose: Provides the application with a more fault tolerant and stable operating system.
Backout: Roll back to 2.7
Contact(s): Wilford Stevens 713-853-7564

Impact: CORP
Time: Sat 9/15/2001 at 8:00:00 AM CT thru Sat 9/15/2001 at 3:00:00 PM CT
Sat 9/15/2001 at 6:00:00 AM PT thru Sat 9/15/2001 at 1:00:00 PM PT
Sat 9/15/2001 at 2:00:00 PM London thru Sat 9/15/2001 at 9:00:00 PM London
Outage: Maintenance on Unix server 'Refraction'.
Environments Impacted: Corp
Purpose: Addresses a known CPU bug and provides additional redundancy.
Backout:
Contact(s): Wilford Stevens 713-853-7564

Impact: CORP
Time: Sun 9/16/2001 at 8:00:00 AM CT thru Sun 9/16/2001 at 3:00:00 PM CT
Sun 9/16/2001 at 6:00:00 AM PT thru Sun 9/16/2001 at 1:00:00 PM PT
Sun 9/16/2001 at 2:00:00 PM London thru Sun 9/16/2001 at 9:00:00 PM London
Outage: Maintenance on Unix server 'Fracture'.
Environments Impacted: Corp
Purpose: Addresses a known CPU bug and provides additional redundancy.
Backout:
Contact(s): Wilford Stevens 713-853-7564

Impact: CORP
Time: Sun 9/16/2001 at 10:00:00 AM CT thru Sun 9/16/2001 at 4:00:00 PM CT
Sun 9/16/2001 at 8:00:00 AM PT thru Sun 9/16/2001 at 2:00:00 PM PT
Sun 9/16/2001 at 4:00:00 PM London thru Sun 9/16/2001 at 10:00:00 PM London
Outage: Maintenance on Unix server 'Emerald'.
Environments Impacted: Corp
Purpose: Addresses a known CPU bug and provide additional redundancy.
Backout:
Contact(s): Wilford Stevens 713-853-7564

Impact: CORP
Time: Fri 9/14/2001 at 8:00:00 PM CT thru Sun 9/16/2001 at 2:00:00 AM CT
Fri 9/14/2001 at 6:00:00 PM PT thru Sun 9/16/2001 at 12:00:00 AM PT
Sat 9/15/2001 at 2:00:00 AM London thru Sun 9/16/2001 at 8:00:00 AM London
Outage: CPR
Environments Impacted: All
Purpose: CPR Database Migration
Backout:
Contact(s): CPR Support 713-284-4175

Impact: CORP
Time: Fri 9/14/2001 at 8:00:00 PM CT thru Fri 9/14/2001 at 8:30:00 PM CT
Fri 9/14/2001 at 6:00:00 PM PT thru Fri 9/14/2001 at 6:30:00 PM PT
Sat 9/15/2001 at 2:00:00 AM London thru Sat 9/15/2001 at 2:30:00 AM London
Outage: Break The Third Mirror on Skywalker
Environments Impacted: ENA
Purpose: Reduce fallback time if needed for ERMT database upgrade.
Backout: Reattach the mirrored disk.
Contact(s): Alex Hsieh 713-853-6561 713-288-1779

Impact: EBS
Time: Sat 9/15/2001 at 7:00:00 AM CT thru Sat 9/15/2001 at 3:00:00 PM CT
Sat 9/15/2001 at 5:00:00 AM PT thru Sat 9/15/2001 at 1:00:00 PM PT
Sat 9/15/2001 at 1:00:00 PM London thru Sat 9/15/2001 at 9:00:00 PM London
Outage: Migrate EBS Databases GeoQual, LiveLink, LMS, and Remedy
Environments Impacted: EBS
Purpose: Migrate EBS Databases into ENW infrastructure environment
Backout: Switch back to previous servers
Contact(s): Scott Williamson
Ed Belcher 713.345.5502 713.306.7379

SITARA:
Impact: sitara production
Time: Sat 9/15/2001 at 6:00:00 PM CT thru Sun 9/16/2001 at 6:00:00 AM CT
Sat 9/15/2001 at 4:00:00 PM PT thru Sun 9/16/2001 at 4:00:00 AM PT
Sun 9/16/2001 at 12:00:00 AM London thru Sun 9/16/2001 at 12:00:00 PM London
Outage: Sitara Production for OS upgrade
Environments Impacted: Sitara users
Purpose: System Maintenance
Backout: revert back to Sun OS 5.7
Contact(s): SitaraonCall 713-288-0101

SUN/OSS SYSTEM: No Scheduled Outages.

TELEPHONY: No Scheduled Outages

TERMINAL SERVER: No Scheduled Outages.

UNIFY: No Scheduled Outages.

SCHEDULED SYSTEM OUTAGES: LONDON
Impact: CORP
Time: Fri 9/14/2001 at 6:00:00 PM CT thru Sat 9/15/2001 at 9:00:00 PM CT
Fri 9/14/2001 at 4:00:00 PM PT thru Sat 9/15/2001 at 7:00:00 PM PT
Sat 9/15/2001 at 12:00:00 AM London thru Sun 9/16/2001 at 3:00:00 AM London
Outage: Complete Powerdown of the London office
Environments Impacted: All
Purpose: Installation of a third generator at Enron House
Backout: Switch all equipment back on.
Contact(s): Tracy Pearson 830-34238 London Tie Line


----------------------------------------------------------------------------------------------------------------------------
FOR ASSISTANCE

(713) 853-1411 Enron Resolution Center


Specific Help:
Information Risk Management (713) 853-5536
SAP/ISC (713) 345-4727
Unify On-Call (713) 284-3757 [Pager]
Sitara On-Call (713) 288-0101 [Pager]
RUS/GOPS/GeoTools/APRS (713) 639-9726 [Pager]
OSS/UA4/TARP (713) 285-3165 [Pager]
CPR (713) 284-4175 [Pager]
EDI Support (713) 327-3893 [Pager]
EES Help Desk (713)853-9797 OR (888)853-9797
TDS -Trader Decision Support On-Call (713) 327-6032 [Pager]