Book Review: The Phoenix Project

I’ve broken this post up into two parts, the first directed at convincing you to buy this book and read it several times, and the second to open up discussion for those who have read the book. There will be spoilers in the second part.

PART 1: No Spoilers

Link to Buy on Amazon

I borrowed this book from a co-worker on Friday and finished it Saturday. Yup, done in one day. 382 pages of stories that seem like they could have come straight from my work related nightmares.

The main character Bill takes over after his boss and his boss’s boss both leave the company. The company is not an IT company and the growing complexity of IT has caused great stress and financial loss.

It is an obvious plug for DevOps. By the end of reading you might wonder if there is any other way to get things done. Keep a skeptical view and enjoy this book.


After the first 10 chapters, I didn’t know how much more I could take. I was physically stressed after reading about the constant firefighting, poor communication, late nights, political sabotage, yelling, swearing, night/weekends/all-nighters, and unreasonable demands. The book depicted a sad state of affairs. I recognized some of the outages, and even the blame game comments sounded spot on.

Its like they consolidated the most frustrating parts of my 9 years at my current company into 3 months. I’m a SAN administrator and that first outage of payroll that got blamed on the SAN but ended up being a poorly implemented security feature caused my first wave of stress. It was like watching a horror movie. “corruption” is like the catch all for unknown software errors. If you take action based on wild assumptions, bad things are going to happen. And let me tell you they continue to happen even though the new boss Bill seems to have a calm logical approach to things.

I wonder if this book was written like Dilbert, where the author was simply writing about what really happened to him. Its the only way this could be so close to accurate.

About halfway through the book, I had a guess that 3 of the secondary characters that were helping Bill, especially Erik, may have just been his alternate personalities. Wes is the aggressive obnoxious one, Patty is the over documenter and process type, and Erik is philosophical one. I was actually disappointed that they remained real characters and not imaginary. I think it would have added to the story to find out that Bill had really just been going crazy from all the stress.

Change Control

I loved watching the team be shocked at how many changes actually happen in the ops world that they have been living in. How could they not know? Changes are like queries on a database, sometimes it makes sense to count them, but mostly they are so different that they can’t be counted. One single big change can be more impactful and riskier that 1000 small changes combined.

Who changed what, when? Questions all ops teams should be able to answer. The book describes “changes” as one of four types of work. I’m not really certain how it fits into DevOps. Maybe change control is about reducing unplanned work, which is another type of work.

I liked the compromise they made between using the crappy change control system, but still forcing and encouraging teams to communicate by writing them on cards. It started a habit and the process communicated the vision. It was an early win in their struggles. The system had so many side benefits such as discovering the Brent bottleneck.

I wouldn’t encourage IT departments to use an index card method to schedule changes. Its not searchable and doesn’t scale well. A heavy handed software application with too many required fields is not the best approach either. The key is having clear definitions of what “Change” really means and what systems need to be tracked the most. IE: important financial systems such as payroll.


This concept hit close to home. My team has lost two people in the last few months and the workload is climbing to unprecedented levels. The automation I’ve put in place is in need of upgrades and important business projects are coming to fruition.

When you are busy, you make mistakes. When you make mistakes, its time consuming to recover. You also take shortcuts that tend to create more work in the long run. Being busy sucks the life out of people.

Decreasing the wait time for IT to add value to the business is was DevOps is all about. The book illustrates this quite well across several fronts. The way Bill achieves some of his goals before achieving kumbaya in the datacenter is with endless hours. He gets denied more people so he takes his salaried workforce and makes something out of nothing.

The graph describes why wait times go through the roof. People can function quite well until they are over 90% busy, from there wait times go through the roof. You can’t squeeze 11% of output out of 10% of idle time. It creates context switching penalties and queuing. This drives the wait times through the roof.

This is why I sometimes work long hours. I know that if I fall behind, it piles up like laundry and I have no clean underwear. It didn’t quite click until I saw the graph in this book but it make total sense. Trying to squeeze that last little bit of production out of a person or process can lead to devastating results.

In the book, Bill realizes he needs to dedicate Brent to project Phoenix. I like the pool of people dedicated to dealing with escalations that usually go to Brent. Its like training without the training. Allowing Brent to focus leads to some interesting automation discoveries later in the book.

Everything is Awesome!

After the first 10 chapters, the book slows down its pace quite a bit. Some characters turn a 180 and everything starts going better. It was a little harder to read and the politics started to take over.

The authors started to apply DevOps approaches to a small team and everything just magically worked. I was hoping there would be continuing issues before they actually got things right but magic pixie dust just made things work. Brent’s server builds just converted over to the cloud without mention of problems or massive costs increases that they already sunk into onsite servers not to mention the architectual shift that would have had to take place to successfully run in the old code in the cloud. But I suppose they were close to 10 deployments a day so it would have been fast right?

Leave a comment

Posted by on November 12, 2015 in Virtual


SQLPass Day 2,3,4,5,6,7

Definitely thought I was going to take some time out of each day to recap. Turns out that was a little ambitious.

SQLKaraoke Sunday night was my first experience with Bush Garden. If you are ever in a situation where you are trying to decide to be the 5th dude in a cab to go sing until morning, just say yes. Mrs. Bush says its ok. Thats all I have to say about that.

See, no 's'

See, no ‘s’

Monday was a recovery day. I transferred from my AirBnB to the Sheraton. $199/night was a fair rate for a nice hotel right next to the conference center.

Tuesday was my pre-con. I originally had talked with a co-worker about what pre-con we were going to attend and I recommended he check out Kimberly’s session since I have seen a small condensed version already. I attended the enterprise scripting workshop by the Midnight DBAs.



The sessions I attended were:

SQL Server 2016 on Flash
SQLCAT: SQL Server HA DR Customer panel


Overview of Azure SQL Data Warehouse

The Plan Cache Whisperer

What a strange room for Jason's presentation...

What a strange room for Jason’s presentation…

Whats new In Reporting Services 2016

Datazen baked right into Reporting Services, lots of applause

Datazen baked right into Reporting Services, lots of applause

Inside Wait Types, Latches, and Spinlocks

SQL Server on Flash: Re-Thinking Best Practices

Part of Glenn’s DMV talk
Analysis Services: Show Me Where It Hurts
Advanced Indexing
Datazen Technical Depp Dive

Advanced indexing was a great session, but I think I have to give a slight edge to Jason’s Plan Cache Whisperer if I had to pick a favorite. Jimmy is also a great presenter and he had lots of the information on modern flash I was looking for.

Leave a comment

Posted by on November 6, 2015 in PASS


SQLPass Day 1: Exploring Seattle before #summit15

This first one is some boring route talk I’ll admit.

Alaskan Air from DTW to SEA non-stop round trip for $315. Pretty good find. I decided to fly out a couple days early because I know there will be some other SQL people I might be able to meet up with, and the flight times were pretty terrible otherwise. I’m a SQL PASS Summit first timer and this will also be my first time to Seattle. I want to checkout the normal touristy stuff like the fish market, coffee shops and the space needle. The week is pretty busy with other activities so the extra days should be more relaxing and fun.

I’m about a 1 1/2 hour drive from the airport so I opted for the Michigan Flyer bus for $50 round trip instead of gas and parking that would be a little bit more expensive.

MichiganFlyer driving past the Blue Lot

MichiganFlyer driving past the Blue Lot

My wife and I tried out AirBnB while in Europe and found it pretty interesting. The first two nights are more of a vacation than a technical training conference, so I offered to pay for the extra costs of food and stay for these nights instead of suggesting work pay for that. I wanted to stay somewhere close to the attractions so I could walk and hotels in that area were about double the price of reasonable looking AirBnB rooms.

Mount Something... Must be getting close

Mount Something… Must be getting close

I’m staying in the Queen Anne district right near the space needle. I took my first Uber from SeaTac to where I am staying for $29. I’m not sure if that is good or bad but I didn’t even realize I had actually ordered the cab until he was calling and saying he was there. Worked out quite well, 5 stars.

My AirBnb host isn’t here but she had someone clean up the place for me. It is a pretty fancy 1 bedroom condo with a view of the space needle.

I got in town and started walking from bar to bar watching the twitter for some meetup opportunities. No luck the first night, but I’m hopeful for a full day tomorrow.

Just the Tip

Just the Tip

Leave a comment

Posted by on October 25, 2015 in PASS


The many different approaches to performance tuning SQL Server

Ever since we started the GLASS user group this spring, I’ve had the idea that we would have a lightning talk style meeting. This is where we have several shorter presentations instead of one long one. My goal was to get newer speakers a chance to dip their toes in the water and help build on a full session that they could present later.

Everyone has a different approach to tuning SQL Server. Different is good, at least on this topic. There can be a lot of friction when trying to troubleshoot where the slowness is happening especially when an organization has a lot of silos. If the tier 1 support has to talk to tier 2 support who has to talk to a developer who has to talk to a server admin who has to talk to a dba who has to talk to a storage admin who… you get the point. I want to get as many perspectives of real world solutions to performance problems together in the same room. Some may think of it as a WWE style smackdown but I think the collaboration would be insanely beneficial.

I couldn’t have been more right :]

We had Kyle talk about implicit conversions specific to SSRS, Mike talk about partitioning, Dave talked about the optimizer, Tom talked about the speed of a single DECLARE or multiple DECLARE statements and I wrapped it up with performance triage with metrics, queries and real world fixes.

The performance tuning process is just that, a process, not a single answer to a problem. There are several ways to approach slowness of an application, and it depends on the situation of how you proceed. Dive right into active queries? Look at the VM CPU graph? Fire back with a bunch of questions? I’ve personally taken all of these angles and found some successes, and a bunch of failures along the way.

Leave a comment

Posted by on September 20, 2015 in PASS, SQL Admin, SQL Dev


Cisco Live 2015 San Diego Recap

This was an impressive conference! Photo Alubum

I flew in Sunday and enjoyed a day getting familiar with sunny San Diego and getting a good nights rest before my test Monday morning. I passed the CCENT exam after a grueling month of preparation. I’m not sure what direction I will take with my Cisco certifications, but the CCNA Data Center track looks appealing and has some overlap with what I have already learned.

The number and quality of sessions makes me consider this conference one of the best I have attended. I’ve been to VMworld, EMC World, SQL Rally and SQL Connections and this one ranks at the top for overall quality. I’d recommend it to anyone remotely close to managing a network.

I focused on storage networking, security and UCS for the sessions I attended. I was able to get some time at the whiteboard with a Fiber Channel expert who helped me walk through a possible upgrade path. In the storage networking sessions I had some interesting discussions about flash arrays with my peers. Looks like a lot of people are getting into testing “seed” units that were provided for free.

The conference food was just ok but the exibit hall had some good appetizers and drinks. The Gas Lamp Quater is a hotbed of excellent restaurants including Fogo De Chao which well worth my $70 for dinner. The entertainment was great. OK GO opened up the conference keynote and Areosmith rocked Petco park. Mike Rowe had some hilarious stories and a good closing message.

I like to travel and learn about technology. Its always re-invigorating to attend a conference and I hope there are many more in my future.

Leave a comment

Posted by on July 3, 2015 in Uncategorized


ICND1 100-101 Study Progress 2

I have reached page 682 of the Odem book which is where I am going to stop. Now I am going finish typing up my notes. Next I will use the attached CD to quiz myself to figure out what areas I need to brush up on in the coming weeks.

CHAP19 Subnet Design p533
– count the bits know the powers of 2
– 2^10 is 1024 and that is easy to remember

CHAP20 VLSM p561
– Old routing protocol doesn’t support vlsm (RIP)
– no additional config to get this to work
– be able to find overlap of networks to troubleshoot

CHAP21 Route Summarization p577
– strategy used for performance to lower the size of routing tables
– subnet design should have summarization in mind
Steps to finding the best summary route
1. list all decimal subnets in order
2. note low and high points
3. pick the shortest prefix length mask and subnet -1
4. calculate new potential network mask summary

CHAP22 Basic ACLs p599
– ACLs most common use is a packet filter
– can match source and/or destination
– match packets for QoS
– to filter a packet you must enable acl on the interface either enter or exit
– NAT uses ACL permits
– when processing ACL list router uses first match logic
– ex command: access-list 1 permit
– To figure out wildcard, get mask and subtract

*know where the best place to put the ACL is and on what router in the path

CHAP23 Advanced ACLs p623

ACLs are numbered or named
– to make a change to the list, must delete the whole list and reconfigure
– extended ACLs allow for more packet headers to be searched
– example command: access-list 101 permit protocol SIP wildcard DIP wildcard
– example command: access-list 101 deny tcp any gt 1023 host eq 23
– keywords can be used instead of port #s (HTTP instead of 80)

Named ACLs, differences
– easier to remember
– subcommands not global
– allows single line deletion

numbered ACLs allow for new style of command

config t
do show ip access-list 24

– use the “enable secret” command
– username secrets if external auth not available
– disable telnet
– avoid using simple password checking
– disable unused services
– use ACLs to secure SSH
– extended ACLs close to source
– Standard ACLs close to destination
– Specific ACLs early in list

enable secret myPass
-this sets the password of myPass to reach enable mode

CHAP24 NAT p653
– CIDR route summarization
– classless interdomain routing
– inside local: local ip assigned to host
– inside global: what the internet knows your network as. address used to represent inside host as packet hits internet
– outside global: public ip outside enterprise (the ip of the URL you are trying to access)

PAT is port address translation
pic on p664-uses source port to return traffic to proper client
NAT troubleshooting
-don’t mix up ip nat inside and ip nat outside addresses
-don’t mix up local and global addresses in this command: ip nat inside source static
-dynamic NAT uses ACLs, check these
-PAT uses the overload command on ip nat inside source command


I took a couple 10 question tests from the CD. The idea was hit some chapters that I struggled with which were, WANs, ACLs and NAT. I got 6 out of 10 questions right which isn’t all the great.

Next I took a test of the first 5 chapters of the book. I scored 8 out of 10 right which is passing for the book test. The only concept I wasn’t sure on was crossover cable pin numbers and when to use a straight through and crossover cable. I knew like devices use crossover cables but that alone didn’t help me get the two questions right. I may memorize this table for the test.

routers Hubs
pcs Switches
1,2 3,6
Leave a comment

Posted by on May 30, 2015 in Network Admin


ICND1 100-101 Study Progress


I’m starting to see the fruits of an aggressive study plan. Here we are, May 23rd, roughly two weeks until test time and I am nearly on track.

Part I: Networking Fundamentals
Part II: Ethernet LANs and Switches
Part III: Version 4 Addressing and Subnetting (Be done by May 11th and practice subnetting)
Part IV: Implementing IP Version 4 (Be done by May 18th and practice show commands)
Part V: Advanced IPv4 Addressing Concepts
Part VI: IPv4 Services (Be done by May 26th, decide if I want to skip IPv6, Review OSPF and practice more advanced subnetting)
Part VII: IP Version 6
Part VIII: Final Review (Be here by June 1st and have taken a practice exam to decide what areas to review)

I got off to a rocky start with an older 2008 version of the book. Fortunately my study buddy had purchased the correct book instead of borrowing an old one. I had gotten two chapters into the old book and before I started to really get into the newer edition that took a week to recieve. I decided to take a practice test early on. The test is very configurable. I chose study mode for 45 questions and limited myself to 90 minutes with a small chunk of whiteboard. I also decided to exclude any IPv6 questions from this first stab.

After two chapters and a couple videos on subnetting I was able to get a 600 which is 200 points away from passing. This was on the practice test that came on the CD in the book. The higher layer concepts I did quite well on where as the lower layer concepts such as Routing, WANs, ACLs and any kind of IOS commands and configuration questions I did very poorly on. Subnetting seems to get a lot of attention either directly, or indirectly and I was sitting at about 50% or less on that.

What is subnetting?

Don’t listen to me, I’m not an expert, but I don’t think there are many good explanations of this out there. A lot of people go way deep and off on tangents to frequently. Here is my overview of what I understand are important subnetting concepts for ICND1.

IP Address = 32 bits = 4 Octects = 4 bytes

Each byte can store 256 possible combinations of 1s and 0s. So lets represent in binary, 00001001.00000000.00000000.00000001

See, that is 32 bits in an IP address.

The second concept we need to understand is the netmask. Picture a mask you might put on your face. A very thick mask you won’t be able to see much. A thin mask you might be able to see a lot.

Take that concept and apply it to this very common netmask, or 11111111.11111111.11111111.00000000

Out of all the possible combinations that is a pretty thick mask so I can only see a small number of hosts with that mask. If you combine the IP & netmask, you will be able to see IP address from or 256 possible hosts.

And there you have it, networking. Wait, what was I talking about? Ah yes, SUBnetting.

Subnetting takes those 256 possible hosts and divides them into smaller networks. If I needed several separate networks and only 18 hosts per network I could split that network into smaller chunks. If I want to see fewer hosts in my network I need a thicker, or higher number mask.

Pulling up the /24 mask again, 11111111.11111111.11111111.00000000 you will see it is /24 because there are 24 1s or network bits and 8 0s or host bits.

In our problem, we need at least 18 IP address options for hosts. For this we will use 0s. How many 0s will we need? Less than 8 for sure because that gave me 256 options. But how many less?

The powers of 2 come in handy for any binary math. There are 2 possible values for each bit, 0 or 1. With 2 bits there are 4 possible values, 00, 11, 10, 01. That isn’t going to get me to at least 18 hosts. This could take a while and for the ICND1 test you need to subnet in 15 seconds. Yikes!

In comes the cheat sheet.


Memorize this formula to go with the table: Possible hosts on a network = 2^h – 2

Each network supports 2^h ip addresses, however 1 ip address is used for the network id and another is used for the broadcast address, hence the minus 2 part.

I don’t suggest just memorizing the table. I would suggest understanding how to generate the table. Start from the top right and do your powers of 2 up to 128. 2^0 = 1, 2^1 =2 2^2=4 … 2^7=128

Next is the second row, the decimal mask. Take 256 – the h row to get the decimal mask row.

Next is the last 2 octets of cidr notation. This is simply a count of 1s in the binary representation of the mask. Remember 1s are the network bits and 0s are the host bits.

Once we have this table we can solve our problem, subnet in a way that supports at least 5 networks and at least 18 hosts in each network.

Start this question with the important number h, or 18.

Go to the table and find the h value that supports at least 18 hosts, which is 32.

Go down to the decimal notation .224 and we know that we can support at least 18 hosts with a decimal mask of

Next we can list the network IDs that this mask could possibly create

To figure this out mathematically take 2^n where n = the number of network bits. There are 3 network bits or 1s in the octect we subnetted. We can make 8 networks which is greater than 5 required by the problem. BOOM CAKE!


For the remainder of this post I will be simply typing up my notes from the Wendell Odom Cisco Press Book and some other notes I took watching YouTube videos from a variety of authors which I will link to.


1. Physical – wiring standards, physical topology, bandwidth usage, syncronizing bits
2. DataLink – MAC, Flow Control standards
3. Network – IP, IPX, Switching, Route Discovery, TTL
4. Transport – TCP, UDP, windowing, buffering
5. Session – Netbui
6. Presentation – jpg, encryption, data formatting(ascii, ebcidic)
7. Application – http, smb, smtp, service advertisement, dns

IP Addressing

First Octects
CLASS A – 1-127
CLASS B – 128-191
CLASS C – 192-223

Hub – layer 1 device that simply spams all ports with frames

Rember these things in this order
SEGMENT – includes the tcp ports
PACKET – includes the IP
FRAME – the whole stinking thing with headers and trailers

Encapsulation – IP Packet is a Layer 3 PDU

CHAP2: Fundamentals of Ethernet Lans

UTP – unshielded twisted pair

crossover cable

like devices need crossover cable to switch transmit and receive pins

MAC – 48bits – 24 for OUI

FCS – frame check sequence is at the end of the frame to ensure proper delivery


leased line , service provider
CPE – customer premises equipment
CSU/DSU – channel service unit, data service unit usually on prem and RJ-48
Router-Router communication can occur on serial cables
HDLC – high level data link control
——way of encapsulating frames over WAN
PPP – point to point protocol
MPLS – multi protocol label switching

CHAP4: IPv4 Addressing and Routing

Routing uses L3PDUs
Layer 2 are called frames

IPv4 headers are 20 bytes and include SIP,DIP,Len,offset,chksum,ttl,etc…

CLASS A: 126 networks and 16,777,214 hosts per network
CLASS B: 16,384 networks and 65,534 hosts per network
CLASS C: 2,097,152 networks and 254 hosts per network

Router FWD logic
1. uses FCS to make sure no errors
2. discard old frame header and trailer
3. compare DIP to routing table and find next hop
4. encapsulate

CHAP 5: fundamentals of TCP/IP transport applications

UDP – connectionless

Connection establishment
SYN —->

Connection termination

shutdown – command that turns a port down/down
no shutdown – turns a port up/up (the second up is if the protocol works)

CHAP 8: configuring Ethernet Switching

enable secret mYpass
show history
no shutdown

port security
1. switchport mode access (access or trunk)
2. switchport port-security (enables port security)
3. switchport port-security maximum 2 (allowed macs on port)
4. switchport port-security violation shutdown (action to take)
5. switchport port-security mac-address AAAA:AAAA:AAAA (specifiy allowed macs)
6. switchport port-security mac-address sticky (dynamic learned mac addresses)

CHAP 9: implementing VLANs

ISL = OLD protocol

12bits for VLANID (this is a “shim” in the frame)
how many vlans? 2^12 or 4096
vlanid 1 is default

router on a stick – one physical link to a router instead of two

show vlan brief

(allow port 4 to communicate on vlan id 10)
1. enable
2. configure terminal
3. interface FastEthernet0/4
4. switchport access vlan 10

Layer3 switch does routing …but can’t do this in packettracer :[

Reasons switch prevents VLAN traffic from crossing a trunk
1. removed from allow list
2. vlan doesn’t exist in show config
3. vlan doesn’t exist, been disabled

and some other less important reasons

CHAP 10 Troubleshooting

show cdp neighbors
show interfaces status

“administratively down” means shutdown command was run
err-disabled means port security

vlan troubleshooting
1. identify all access interfaces and their vlans
2. do vlans exist and are they active
3. check allowed vlan list on both ends of the trunk
4. check for trunk/no trunk neighbors

show vlan brief


IPv4 subnetting

One subnet for every:
1. vlan
2. ppp serial link
4. frame relay

VLSM – variable length subnet mask


CHAP 12 analyzing classful IPv4 Networks
CHAP 13 analyzing subnet masks
CHAP 14: analyzing existing subnets

CHAP 15: Operating Cisco routers

Installation steps
1. connect lan ports
2. connect CSU/DSu external
3. connect CSU/DSU internal
4. connect console port to pc using a rollover cable
5. connect power
6. power on

show ip route

show mac address-table

status layer 1/status layer 2
down/down : has not been shutdown but physical layer problem

CHAP 16: configurating IPv4 addresses and routes

1. choose to process frame
-proper mac (is its destination me?)
-no errors (FCS)
2. de-encapsulate packet
3. compare DIP to routing table
-this identifies outgoing interface
4. encapsulate
5. transmit

routers should ignore switch floods not intended for it

large routing tables can cause performance problems

cisco express forwarding
-uses organized tree and other tables to help speed up routing

adding routes can be done via:

1. connected routes
2. static routes
3. routing protocols

cisco will add routes if the interface is IP’d and UP

ROAS 802.1Q trunk


commands to turn on

router ospf 1
network area 0

ospf – open shortest path first – uses link state
OSPFv2 is for IPv4

routing protocol – set of messages, rules and algorithms (RIP, EIGRP,OSPF,BGP)

routed & routable protocol – defines packet structure and addressing (IPv4)

1. learn routing information about ipsubnets from neighboring routers
2. advertise this info
3. if more than 1 route exists, pick best
4. if topology changes, advertise current best route (convergence)

Interior gateway protocol – designed for use inside a single autonomous system
exterior gateawy protocol – BGP

routing algorthims use
1. distance vector
2. advanced distance vector
3. link state (ospf uses this)

RIP is old
IGRP is a little less old

RIP-2 uses hop count and is also old with slow convergence
OSPF is a cost based protocol
EIRGP – cisco proprietary and uses bandwidth and latency
IS-IS – uses link state

0 connected
1 static
20 BGP E
110 OSPF
115 IS-IS
120 RIP
200 BGPI

this will show the database of link state advertisements(LSAs)
show ip ospf database

routers must agree to be neighbors

configuration, this will turn on for any interface that matches 10.0.* because of the wildcards in network command

router ospf
network area 0


Discover – TO FROM

ip helper-address {dhcp server ip} – command for router that enables DCHP servers to sit outside of the subnet by changing SIP&DIP ( Thanks /u/Sprockle )

Leave a comment

Posted by on May 23, 2015 in Network Admin


Gearing up for another exam ICND1 100-101

The more I learn about networks, the less I tend to blame the network.

It was almost 20 years ago that I set a static IP address on my sisters computer and connected a cross over cable to my computer so we could play a game called Quake. She wasn’t that interested so I ran back and forth between the rooms and played by myself. This loneliness was resolved a few years later with a device that looked something like this


Point is, I’ve been doing this for a long time and I still don’t know jack. I don’t like to fail tests, so signing up for a test is going to help me learn. I would like to become a more well rounded datacenter administrator.


ICND1 100-101 is the first half of a valuable certification CCNA. I now have the book in hand and about 5 weeks to prepare. Normally, I would allow myself about 3 months with a book this size but opportunity has struck and I need to accelerate my pace.

Like Microsoft, Cisco is very open with their exam topics.

1.0 Operation of IP Data Networks 6%
2.0 LAN Switching Technologies 21%
3.0 IP addressing (IPv4/IPv6) 11%
4.0 IP Routing Technologies 26%
5.0 IP Services 8%
6.0 Network Device Security 15%
7.0 Troubleshooting 13%

These do not line up that nicely to the book topics. But I am going to attempt to cruise through the book which I have given myself some milestones below.

Part I: Networking Fundamentals
Part II: Ethernet LANs and Switches
Part III: Version 4 Addressing and Subnetting (Be done by May 11th and practice subnetting)
Part IV: Implementing IP Version 4 (Be done by May 18th and practice show commands)
Part V: Advanced IPv4 Addressing Concepts
Part VI: IPv4 Services (Be done by May 26th, decide if I want to skip IPv6, Review OSPF and practice more advanced subnetting)
Part VII: IP Version 6
Part VIII: Final Review (Be here by June 1st and have taken a practice exam to decide what areas to review)

The schedule is set, plans are in place, now it is time for me to do some reading.

Leave a comment

Posted by on May 9, 2015 in Network Admin


T-SQL Tuesday #065 – Slowly Changing Dimensions

tsql2sday150x150_thumb_2aa4ea0f I’ve been focusing a lot of my study time on data warehousing lately. I’ve been supporting the system and storage of data warehouses for a while but lately have been digging into the developer topics.

What I learned over the weekend is how to build a working, slowly changing dimension in SSDT. Thanks for the challenge #tsql2sday and @SQLMD!


The Problem

Dimensions are the tables we design to make data look good in a pivot chart. They are the tables that describe our facts. Customer is a good example of something that could be a dimension table. For my challenge I decided to use virtual machine as my dimension.

The problem is, what if a VM’s attributes change? 4 cores, that was yesterday.. today PRDSQLX has 24 cores. What if someone deletes a VM, how many cores did it have?

I can get the current status of my VMs by using the source system, but the problem is the history. I can pull a snapshot of what VMs I have in my environment every day from the source system. I could just make copies of that data and slap a “PollDate” column on the table. Viola, I have everything I need, and about 1000x more than I need.

There is the problem, how do I collect and save a history of my VM’s attributes?


Each column in my VM table can be of 3 basic types

Type 1. Simply overwrite this value… it changes a lot and I don’t care about history (eg. what host is the VM running on)
Type 2. add a new row to maintain history… if one column in my VM row changes, I get a whole new record in my dimension
Type 3. add a new column to keep a limited amount of history… add some columns like previous_num_cpus and previous_previous_num_cpus and move data to that as it changes

So we have to take the data we get on a nightly snapshot of the source, and compare it to what we have in the destination, then do a conditional split. I’m sticking to handling these differences:

New VM – insert with NULL validto (easy)
Deleted VM – change validto column (create staging table and do an except query)
Change in Type 1 Col – update existing VM row with NULL validto column, (easy)
Change in Type 2 Col – insert new row with NULL validto column, change previous record’s validto date (a little tricky)

That logical split can be made easier by using the Slowly Changing Dimension task in SSDT. It pops up a wizard to help you along the way and completely set you up for several failures which I am going to let you learn on your own :]

Step 1. Setup an initial loading package.

This will make it handy to restart your development.

Query the source in a data flow OLE DB Source
Tack on a few extra columns, validfrom, validto, isdeleted, sourcesystemid in the SQL command
create the destination table using the new button ( this is pretty handy to avoid manually lining up all datatypes )
use the new button again to create a dimVM_staging table for later
Add the task at the beginning of the control flow to truncate destination or dimVM table
Run the package and be careful not to accidentally run it since it has a truncate

Step 2. Create this monstrosity

Control Flow

Data Flow

It is actually not too terribly bad. When you add the Slowly Changing Dimension a wizard pops up and when all the stars align, all the data flow transformations and destination below are created.

If we focus on the top of the data flow first, it is easy to see I am pulling from two source systems and doing a union all. The interesting problem I had to solve was the deleted VM problem. The wizard didn’t do that for me. I knew if I had the staging table, I could compare that to the dimVM to see if anything was missing. If you want to find out what is missing, use an EXCEPT query. Once you find out what is missing (deleted VMs) we can update the validto field effectively closing up shop on that row but keeping the history of rows relating to that VM. I decided to add the isdeleted column to make it easier to find deleted VMs. This code is in the SQL Script task on the control flow.

update dimVM
set dimVM.validto = getdate(), dimVM.isdeleted = 1
from dimVM
inner join (
select vmid,vcenter from dimVM
where validto is null
select vmid,vcenter from dimVM_staging
) del
on dimVM.vmid = del.vmid and dimVM.vcenter = del.vcenter

One last little tidbit. If you make any modifications to the transformations that the SCD wizard created, you should document them with an annotation. If for some reason you have to get back into the wizard, it will recreate those transformations from scratch… ironically not maintaining any history.

Step 3. Profit

I hope you enjoyed hearing about my new experiences in the Slowly Changing Dimension transformation in SSDT.

1 Comment

Posted by on April 14, 2015 in SQL Dev


Final Preparation for 70-463 Implementing a Data Warehouse with Microsoft SQL Server 2012

This is a continuation of this post

Two fellow bloggers have been posting more quality information on this test.

When reading the book I skipped over all of the practice sections. I did read the exam tip sections inside of the practice but never actually practiced. I don’t have a lot of hands on experience with SSIS and even less with mds/dqs. I spent about 9 weeks making through the book while skipping the practice and most of the reviews. I probably would have needed an additional 18 weeks to properly make it through all of the practice or lab type sections of the book. Learn one, do one, teach one is my favorite method to mastery but with 2nd shot deadline, I didn’t have a lot of time to prepare.

To supplement, I attempted to find videos on youtube and watched videos on the Microsoft Virtual academy. Both sources were not very demo heavy. What I did find is CBT nuggets that give a 7 day trial. The 70-461 videos that I was able to watch were very high quality, fast paced and demo heavy. This is exactly what I needed at this time. I’d recommend a membership if you have a bundle of money burning in your pocket.

Since my trial was up I decided to type up my CBT nugget notes.

CBT connections managers
control flow -> doesn’t involve data
bottom level are private connection managers, a.k.a package level
right solution explorer is project level connection managers which are global
you can enable/disable sequence containers
precedence constraints, go to properties to define AND or OR logic
copy-> paste package connection managers
delay validation -> doesn’t check structure
email doesn’t have a port option but could purchase add-ins or write your own
fix for NULLs is COALESCE

Data Flow
rows, buffers, pipeline,transformations
raw file -> ssis only -> good for sharing data between packages
raw file -> good for resuming packages
recordset->variable used to loop through
for performance, aggregate at the source since that is blocking
import export col -> for blob data
term matching is like CTRL+F
blocking tasks take lots of memory -> sort, aggregate
partial-blocking -> merge chuncks

Data Quality services
cleansing matching
server is 3 databases
dqs client is used for creating KBs
creating a knowledge base
-open xls sheet -> job title list for KB example
-KB needs a domain, circle with * button is domain
State length of 2 is an example domain rule
composite domain (EX: address which includes city state zip)
reference data source RDS (ex: mellisa data for addresses)
KB’s get published
activity is automatically logged

Implementing DQS
data profiling task in SSDT
-profile types
–null ratio request
–pattern generator RegEx for formatting
–column statistics
-then specify column
Quick profile: runs against all columns
Open data profile viewer
suggested confidence level
corrected confidence level
DQS cleansing task
Job title source job_title _output
jobtitles table
newKB->domain->source column (survivor record)
the table with the + button to add a rule and use the Rule Editor

Implementing MDS
proactive management
people place concepts or things
non-transaction data is good for MDS
includes auditing and versioning
MDS Componenents(Database, config mgr, MD mgr, web service, mds model deploy, excel Add-In)
MDS Objects(Models: the container db, Entities: like tables, Attributes: like columns, Hierarchies, Members: Actual data)
Install requires powershell 2.0 and IIS 7.5, silverlight and a database
has integration with DQS
to deploy packages that contain data must use CLI (deploynew -package “” -model)

Data flow
merge join requires sort -> advanced editor, pick isSorted and the column
MetaData problems: double click on flow and change types
Lookup transformation
-cache connmgrs for re-use
–redirect rows
–multi output popup
slowly changing dimension task (wizard)
fixed attribute fail on change
changing attribute type 1 overwrite type 2 new records (history)
inferred member flag goes in dimension
blocking oledb command
redirect error rows to flat file

executing packages
dtexec.exe is fire and forget style
built-in SPs in ssisdb
catalog.set_obj_param value
restartable packages
-checkoint file
-tracking last successful step in control flow
project properties
-select file name
-set usage never
–if exist
-save checkpoints = true
-set property fail package on failure = true
to test, can set task property to force a failure

Leave a comment

Posted by on April 9, 2015 in SQL Admin, SQL Dev


Get every new post delivered to your Inbox.

Join 167 other followers