USE
IMPROVE
EVANGELIZE
What is Solaris Nevada? Nevada at Build 72
Ken Tomlinson Solaris RPE Engineering Manager Sun Microsystems
USE
IMPROVE
EVANGELIZE
What is Solaris Nevada?
● ● ● ●
The next generation of Solaris A distribution based on OpenSolaris A community of engineers/developers/hackers “Nevada” codebase / “Nevada” distro codename conflated
(Nevada SXDE & SXCR) (S10 Updates)
Belenix (Nevada) Nexenta
2
USE
IMPROVE
EVANGELIZE
72?
● ●
Builds are released every 2 weeks. Build 74 is currently open.
–
As of September 1, 2007, Nevada is about 145 weeks old
● ●
~15,000 bug fixes / RFEs (since S10) At least 170 community bug fix/RFE contributions
3
USE
IMPROVE
EVANGELIZE
Annotations
● ●
Features already integrated Features planned for Nevada in Italics
– –
No guarantees: feature may not complete, or may not make the release This talk does not discuss what is in, or will, or won't appear in Solaris 10 Update releases (there isn't enough time in this talk!)
4
USE
IMPROVE
EVANGELIZE
Participation
● ● ● ● ●
http://www.opensolaris.org http://src.opensolaris.org Free starter kit: http://get.opensolaris.org 51 User groups worldwide 9 country-specific portals
–
{br,cn,de,es,fr,in,jp,mx,pl}.opensolaris.org irc.freenode.net: #opensolaris, #opensolaris-de, #opensolaris-es, #opensolaris-fr, #opensolaris-pl
●
IRC Channels
–
5
USE
IMPROVE
EVANGELIZE
http://get.opensolaris.org
6
USE
IMPROVE
EVANGELIZE
Nevada Themes
●
Continuing Innovation
– – – – – –
System & Networking Performance Observability Virtualization: Zones, xVM, BrandZ Storage software stack Core Networking features Overhauled Installation and Packaging Improve opportunities for participation New x64 and SPARC Platform Support Device Support, Desktop Goodies
●
OpenSolaris Everywhere
– – –
7
USE
IMPROVE
EVANGELIZE
By the Numbers
● ●
●
~15,000 bug fixes / RFEs (since S10) At least 170 community bug fix/RFE contributions Oldest bug fixed (so far):
–
1085341 32-bit stdio routines should support file descriptors >255 (filed 3/4/1992) 1128254 *csh* csh uses gethostname with MAXHOSTNAME=64 instead of 256 Fixed developed by OpenSolaris community member Yann Poupet
●
Second oldest bug fixed:
– –
8
USE
IMPROVE
EVANGELIZE
Improve Participation
● ●
●
53 additional drivers moved from closed to open src OS/Net consolidation (including kernel) buildable with gcc and Sun Studio on SPARC and x86 More open consolidations:
–
X, Freeware (SFW), Documentation, Man Pages, Translations (G11N), Developer Tools
● ● ● ●
http://cr.opensolaris.org/ for posting code reviews Requirements gathering for defect tracking system Switch to Mercurial for SCM opensolaris.org website app (“tonic”) open source
9
USE
IMPROVE
EVANGELIZE
Not Covered This Hour
● ● ● ●
Virtualization (Later today) Storage (Later today) Networking (Later today) Security Technologies (Tomorrow)
... lots of other projects that we didn't have time to mention...
10
USE
IMPROVE
EVANGELIZE
Indiana
11
USE
IMPROVE
EVANGELIZE
What is Indiana?
● ● ●
● ●
●
OpenSolaris binary distribution Community-built Single CD install and network package repository 100% re-distributable A base distribution in which to build Solaris (or other distro's) Focus on user experience, best-of-breed opensource software
http://opensolaris.org/os/project/indiana
12
USE
IMPROVE
EVANGELIZE
What's the roadmap?
●
Release on 6-month cadence
– –
Preview release in October 07 Official release in March 08
● ●
Each release supported for some interval Solaris Next derives from it at some date TBD
– – – –
Branded, polished Slower technology introductions Longer support cycles Seamless migration from Indiana
http://opensolaris.org/os/project/indiana
13
USE
IMPROVE
EVANGELIZE
Packaging: Vision
Open source maintainers for OpenSolaris Myriad of release types opensolaris.org
Improved open source infrastructure and processes allowing easy and successful community participation
Solaris release
sw.opensolaris.org
Software versioning “recipes”
pkgs.sun.com? sw.sun.com
Appliances
Deployer
conary.sun.com? .sw.sun.com
Common software repository
Users evolve at their own pace
Developer
http://opensolaris.org/os/community/install
Custom distributions
14
USE
IMPROVE
EVANGELIZE
Install: Project Caiman
● ●
● ● ● ● ●
Dwarf Caiman – new GUI delivered in 70 Slim Install – single “live” CD, packages from network repository Distribution Constructor – build your own distro Snap Upgrade – LU-style upgrades with ZFS Integrate device detection/HCL reporting Jumpstart/sysidcfg re-based on SMF profiles Full WAN installation support
http://opensolaris.org/os/project/caiman
15
USE
IMPROVE
EVANGELIZE
Install: Project Caiman
16
USE
IMPROVE
EVANGELIZE
System Performance
17
USE
IMPROVE
EVANGELIZE
Scheduling
●
Hierarchical L-group support
– – –
Useful on 4-way+ Opteron Systems plgrp(1) and lgrpinfo(1) tools http://opensolaris.org/os/community/performance/numa Enables multi-level load balancing policies for manysocket, many-core, many-thread systems. Exploits zero-cost of context switch between threads on same core Single run-queue means threads operate as a pool pulling work from the queue.
18
●
Processor Groups scheduler abstraction
–
●
Shared per-core run queues
– –
http://opensolaris.org/os/community/performance
USE
IMPROVE
EVANGELIZE
Virtual Memory Performance
● ●
Large text pages for executables & libraries Large Page Out-Of-the-Box
–
large pages automatically supplied to applications based on segment sizes, alignment and TLBs.
● ●
Large Pages for Kernel Memory on sun4u Fast TLB shootdown on x86
–
Memory unmap substantially faster
http://opensolaris.org/os/community/performance
19
USE
IMPROVE
EVANGELIZE
Cool Performance Hacks
● ● ● ●
SPARC RSA (for SSL, for example) in the kernel is now about twice as fast as before rand(3c), rand_r(3c), malloc(3c), free(3c) all faster Significant malloc(3c) rewhack in the works x86/x64
– – – – –
64-bit division on AMD-64 platforms: 50% faster SHA-1 faster Non-temporal load/store access optimizations for x64 Much faster memmove on x86/x64 strcpy, strncpy, etc. faster on x86
http://opensolaris.org/os/community/performance
20
USE
IMPROVE
EVANGELIZE
Performance/Watt: Project Tesla
● ● ● ●
Speedstep for some Intel processors in 70 E-Star compliance for Sun x86 Workstations Suspend to RAM (ACPI S3) Longer Term:
– –
Powernow & Speedstep (and on MP systems) Not just laptops and desktops. Servers too! Automatic CPU speed reduction Peripheral power management
Reduce TCO by improving performance per KW/H
–
Scheduler improvements
–
http://opensolaris.org/os/project/tesla
21
USE
IMPROVE
EVANGELIZE
Core Kernel
22
USE
IMPROVE
EVANGELIZE
DTrace
● ●
●
●
USDT is-enabled probes DTrace/Zones mashup (dtrace_proc + dtrace_user) Filesystem, X Server, Sysevent, iSCSI, Postgres, mySQL, providers Integration with higher level languages
– – –
Java, php, perl, ruby, python all exist in varying forms python in snv65+ comes with dtrace probes HotSpot JVM Provider built into Java 6 “Chime” unbundled demonstration tool
23
●
Java API for DTrace
–
●
http://opensolaris.org/os/community/{dtrace, observability}
3rd party toolkits evolving (google for “dtrace toolkit”)
USE
IMPROVE
EVANGELIZE
Fault Management
●
●
To date: 25-40% reduction in annual downtime
2005 InfoWorld Innovation Award for “Predictive SelfHealing”
●
Opteron/Athlon64/Turion FMA
– – – – –
ChipKill on by default CPU & Memory error detection Offlining CPUs & Cores, memory retire PCI-E support Works fine on non-Sun Opteron-based systems US-IV+, US-T1, PCI-E I/O
24
●
SPARC FMA
–
http://opensolaris.org/os/community/fm
USE
IMPROVE
EVANGELIZE
Fault Management: Feature Work
● ● ●
SNMP MIB for FMA, Trap generation I/O Retire agent Predictive Self-Healing and Virtualization
–
Distributed fault management with xVM Common architecture for continuous sensor monitoring and analysis Pluggable backends to provide raw sensor readings
●
FMA Sensors/Health Monitors
– –
IPMI, perf. Counters, SMBus, SMART, etc.
–
●
FMA Simulator and Tools
End-to-end scenario design, injection, verification
25
http://opensolaris.org/os/community/fm
USE
IMPROVE
EVANGELIZE
“New Boot”
●
Goal: Simplify boot, add features, improve first impressions, reduce boot time.
– – –
Status: Integrated for x86, SPARC in development Boots RAMdisk “boot archive” Recovery archive supplied No more real mode drivers or configurator! GRUB is used as the booter (with pxegrub for PXE boot) Other OS's detected at installation Direct Boot has replaced use of multiboot
●
X86 Version
– – – –
● ●
Facilitates USB boot, LiveCDs, ZFS booting SPARC new boot provides Install-Time-Update
26
USE
IMPROVE
EVANGELIZE
Desktop and Mobility Support
27
USE
IMPROVE
EVANGELIZE
Mobility Support
●
Wireless command line tool
– –
dladm(1m) (or the now-obsolete wificonfig(1m)) WPA Supplicant Support Atheros chipset driver (ath), Intel 2100b (ipw) Prism II (pcwl), Intel 2200bg/2915abg (iwi) Intel 3945abg (wpi), Cisco Aironet 340/350 (pcan) Ralink RT2500 (ral), RealTek 8180L (rtw) Downloadable NDIS wrapper driver (bcmndis) for Broadcom chips
●
Wireless driver suite
– – – – –
http://opensolaris.org/os/community/laptop/wireless
28
USE
IMPROVE
EVANGELIZE
Device Support: Desktops
● ●
●
● ● ● ●
DBUS and HAL integration Dcam1394 driver on x86, USB webcam support USB auto-magic (plug in your camera, run gtkam) cdrecord, dvd+rw tools bundled Many more audio devices supported Battery monitoring support Tamarack, New Volume Manager
– – –
http://opensolaris.org/os/community/desktop
Replaces vold with more modern implementation Improves removable media security model Integrates smoothly with GNOME
29
USE
IMPROVE
EVANGELIZE
X Windows
●
Tracking Xorg latest (Xorg 7.2 since snv_58)
– – – – – –
64-bit Xorg server Extensions to support Compiz, Looking Glass Xorg for SPARC (basic m64 support only, more planned) Continuously improving Xorg auto-configuration nVidia driver, OpenGL for x86/x64 MESA support
● ●
DTrace provider for Xorg/Xsun/Xnest DRI support for accelerated 3d graphics
– –
Intel i845 to i945, i965 ATI adaptors
●
Migrate client apps and libs to X11R7
30
http://opensolaris.org/os/community/x_win
USE
IMPROVE
EVANGELIZE
Gnome
● ● ●
● ●
Tracking gnome.org (currently 2.18) Improved Look & Feel (“Nimbus” theme) Improved applications: Rhythmbox, Evince, Ekiga (GnomeMeeting), Pidgin, System Monitor, gtkpod StarOffice 8 Basic system administration tools (Users, Groups, Networking)
http://opensolaris.org/os/community/desktop
31
USE
IMPROVE
EVANGELIZE
32
USE
IMPROVE
EVANGELIZE
Gnome (2)
●
Unified printer administration
–
Presto, auto-magic printing support
●
● ● ● ●
Gnome-pilot with USB syncing (and Evolution support) RealPlayer 10 (/usr/bin/realplay) Acrobat 7 (for SPARC) (/usr/bin/acroread) cdrecord, dvd+rw tools Mozilla replaced by Firefox 2, Thunderbird 2
– –
Flash 9 support Javascript DTrace Probes
http://opensolaris.org/os/community/desktop
33
USE
IMPROVE
EVANGELIZE
Network “Auto-Magic” (NWAM)
● ●
Simplify and automate network configuration Phase 0:
– – – –
Improve duplicate address detection Monitors ethernet port, activates DHCP when needed Scans wireless, prompts user, remembers choices/keys One interface active at a time
● ●
Integrate mDNS service discovery Phase 1
–
Establish a mechanism for storing network configuration information in a profile and auto-activating that as needed
34
http://opensolaris.org/os/project/nwam
USE
IMPROVE
EVANGELIZE
Commands Modernization
●
Numerous community contributions to basic utils:
–
exstr, fstyp, mount, logadm, diff3, swap, csh, nawk, file, ls, hostname, cmp, du, df, id, xargs, cut, wc, cat, etc.
● ● ● ●
Ksh93 Integration Project “Path Modernization”: If it fits, put it in /usr/bin Gnu CoreUtils, Automake, Autoconf, /usr/gnu Refreshed freeware strategy
35
USE
IMPROVE
EVANGELIZE
System and Service Management
36
USE
IMPROVE
EVANGELIZE
Service Management (SMF)
● ●
● ●
● ●
● ●
Public manifest class-action scripts More service conversions (gdm, dtlogin, postgres,...) Consistent syslog message on service failure BSM auditing for admin events and config changes Expanded template support Improved manifest import performance, simplified delivery of early boot services Allow property customization in profiles svcs -x diagnosis for inetd services
37
http://opensolaris.org/os/community/smf
USE
IMPROVE
EVANGELIZE
Visual Panels
●
Common system and service management
– –
GUI configuration for Solaris Componentized design for flexibility and reuse
Customers/ISVs can publish configuration Allows creating alternate UIs/consoles Exports Java Management Extensions (JMX) MBean interfaces
–
●
Proof of concept
http://opensolaris.org/os/project/vpanels
http://opensolaris.org/os/project/vpanels
38
USE
IMPROVE
EVANGELIZE
Visual Panels
http://opensolaris.org/os/project/vpanels
39
USE
IMPROVE
EVANGELIZE
Made Better by... You
● ● ● ● ● ●
Get it, Install it: http://get.opensolaris.org File bugs: http://bugs.opensolaris.org Give this presentation to your local user group Join a mailing list on opensolaris.org Attend an OSUG meeting Write some code!
– –
Add DTrace probes to something Write a driver! bluetooth, usb-networking, irda
●
Projects/Communities welcome your participation
40
USE
IMPROVE
EVANGELIZE
Thank you!
Presented by: Ken Tomlinson http://blogs.sun.com/ktomlins Content provided by: Dan Price, Dave Miner, and others http://blogs.sun.com/dp/ http://blogs.sun.com/dminer
“open” artwork and icons by chandan: http://blogs.sun.com/chandan
USE
IMPROVE
EVANGELIZE
Appendix
42
USE
IMPROVE
EVANGELIZE
Networking
43
USE
IMPROVE
EVANGELIZE
Core Network Performance
●
GLDv3
– – –
Dynamic switching between interrupt and polling 10Gbps NIC support Vlan and Trunking support for off the shelf NICs 7Gbps receive with 1500 byte frames! Improve TIBCO perf 70-80% (recv) 90-130% (xmit) Close to 1 million pkts/sec forwarding on Opteron
Near linear trunk scalability: 4x1Gbps NICS → 3.6Gbps
–
●
UDP Performance
–
●
Forwarding Performance
–
http://opensolaris.org/os/community/networking
44
USE
IMPROVE
EVANGELIZE
Core Networking: Clearview
●
Improve integration between key networking technologies
–
IPMP, IP Tunnels, DHCP, Zones, Dynamic Reconfiguration, Network Observability Tools
● ●
Vanity names for network interfaces More snoop-ability:
–
Tunnels, inter-zone traffic, loopback, inside zone
● ●
IPMP IP interfaces represent IPMP groups Unified interface provides dladm(1m) for all interfaces
–
All interfaces: VLANs, aggregations, vanity names
http://opensolaris.org/os/community/networking
45
USE
IMPROVE
EVANGELIZE
Core Networking: Crossbow
●
● ●
Virtualize NICs based on protocol, service or container Flow management APIs and commands Hardware support for flow processing
–
Take advantage of NIC memory partitioning features
●
Enable interesting resource management configurations
– –
Assign VNICs to xVM Domains or Zones Combines with IP Stack Instances for Zones
http://opensolaris.org/os/project/crossbow/
46
USE
IMPROVE
EVANGELIZE
Other Networking Improvements
●
10 Gbps driver support
– – –
Sun Neptune (nxge) Neterion driver (xge) migrated to GLDv3 Chelsio driver (chxge), Intel 10GB (ixgb)
● ● ●
e1000g driver (Intel Gigabit) overhaul DHCPv6 support Public DLPI Library
47
USE
IMPROVE
EVANGELIZE
Data Storage
48
USE
IMPROVE
EVANGELIZE
Open Storage Community
NFS Client NFS Server YANFS
aka WebNFS
CIFS Client CIFS Server
S-QFS Client Shared QFS Server UFS
YANFS Server ZFS
Honeycomb Clients Honeycomb Server QFS
OSD Initiator OSD Target
iSCSI Initiator iSCSI Target iSNS Server
Solaris Volume Manager Availability Suite (volume snapshot and replication) Archive Manager (SAM) Storage OEM Drivers
Legend
Open source Open source candidate
Traffic Manager (MPxIO)
Fibre Channel Framework Disk/Tape Drivers
49
USE
IMPROVE
EVANGELIZE
ZFS
●
● ● ●
An always-checksummed copy-on-write filesystem Unlimited instant snapshots, clones Pooled storage eliminates storage “stranding” Radically simple system administration
–
zpool, zfs commands are all you need to know
● ●
●
Integrated with FMA, Zones, NFS, iSCSI Ongoing work: Performance, delegated administration, pool history, recursive snapshotting ZFS Boot & ZFS aware Install
–
http://opensolaris.org/os/community/zfs
Imagine a world without UFS...
50
USE
IMPROVE
EVANGELIZE
NFS
●
235 MB/sec (1.9Gbs) on x64/10Gbs gear
–
Future: async RPC, request scheduling. wirespeed!
●
●
Coming: 980 MB/sec NFS-over-RDMA (over IB) Full End-to-End NFSv4 ACL support
–
ZFS support for NFSv4 ACLs
● ● ● ●
Mirror Mounts shareadm(1m) command Directory Delegations pNFS (Parallel NFS)
–
Enables horizontal scaling of network storage
51
http://opensolaris.org/os/community/nfs
USE
IMPROVE
EVANGELIZE
Block Storage
● ●
iSCSI initiator support, iscsiadm(1m) iSCSI target daemon (iscsitgtd)
– – –
Can serve raw devices, files or ZFS volumes as LUNs Integration with ZFS command set DTrace provider for observability Native SATA support, Hotplug support PERC 4e/Si, PERC 4e/Di, PERC 4e/DC and MegaRAID 320-2e, 320-2x adapters
●
New SATA Framework
–
●
lsimega driver
–
http://opensolaris.org/os/community/storage/
52
USE
IMPROVE
EVANGELIZE
Availability Suite
● ●
Provides advanced data services at the block level Point-in-Time (PiT) Copy
–
Supports multiple point-in-time copies Multi-hop capability Guaranteed in-order delivery One-to-many/many-to-one replication Synchronous or Asynchronous modes TCP/IP implementation/media independent
●
Remote Mirror
– – – – –
http://opensolaris.org/os/community/storage/
53
USE
IMPROVE
EVANGELIZE
Local OSUG
● ● ●
Local OpenSolaris User Group Meeting in this room, tonight at HH:MM
– –
... ...
54