UX Advantage: Adam Cutler Adam Cutler on Building a Design Studio Culture Within IBM

Jared Spool and Karen McGrane interview Adam Cutler from IBM on building a design studio culture within IBM.

Karen McGrane: You've been in IBM for more than 15 years now?
Adam Cutler: About 15 years.
Karen: About 15 years, that is amazing. What's changed? IBM is a company that has a long history, a long tradition of design. What's been different in your role over the last 15 years? And, what's new with the design studio that you have in Austin now?
Adam: First off, I used to have to fight to even get in the room. When I first started at IBM in 2001, I showed up at my first meeting, and the engineer looked at me and said, "What are you doing here? You're supposed to paint this stuff up on the way out the door. Decorate that."
Jared Spool: Instead of, "Where do you sit?"
Adam: Yeah, exactly. I realized there and then, that part of what was missing was a good articulation about what design really is, and why people should want it. Not to push around people, but to create the pull.

For the first 12 years that I worked at IBM interactive for the consulting arm. I was doing design work for everybody else. I was just a constant proponent of why people should be considering good design, both internally and externally. Really, where it's netted out was, we're the progenitor of that corporate design program, like with Eliot Noyes bringing in Charles Eames, and Paul Rand, [inaudible 01:16] , et cetera, et cetera.
Karen: No pressure?
Adam: Yeah, no pressure. Design was really at the forefront. The original intent was to humanize technology because if you think back, I mean, for most people, their frame of reference is the "Madman" episode where the guy goes crazy and mutilates himself, because the IBM mainframe has been installed in the ad agency. All of these efforts were made to humanize technology.

In the '80s and '90s when IBM lost its way, design was that thing that got forgotten about. In the road to recovery, everything was about pricing and getting it back on solid financial ground. The design, it got lost, so part of what we're doing with IBM design for the past two and a half years has been reconnecting to that design legacy, and then bringing it forward.

For us, it's not about just mimicking or aping the masters that came before us, but what is this look like going into the 21st century, and how do we apply this every day.
Jared: That design legacy...IBM has always been a business that has reinvented itself. It first made calculators, it made mainframes, the typewriter business for a while, it was in the personal computer business and then it got out of the personal computer business. Now, it's building these large enterprise systems. It's always done paying attention to who its customers are. One of the things about IBM is that customers and users were often different groups.
Adam: Correct.
Jared: This design legacy, it was not always about users, right, so you've had to bring that and make that part of the legacy now.
Adam: Absolutely. As we were spinning up the program back in 2013, one of the first conversations that we introduced to the wider body of people was replacing the buyer with the user, because, to your point, the buyer is very rarely the user of the products that we make. There's a mental shift that goes on internally, that you have to get people to look at a different focal point.

There was a very definite progression of getting people to not lose sight of the buyer but to really start paying attention to why the user was the most important. Because, at the end of the day, to humanize it for ourselves and for others, we were saying that nobody sits down and uses an IBM product for 8 or 10 hours a day. They are using multiple things from multiple vendors all the time.

If we can be a part of the change that helps you get home 15 minutes earlier, so that you can make a kid's soccer practice, or spend the time working out or just relaxing, or just making it not suck for that part of your day, that's the intent.

We know that we can't control the whole experience. It used to be that you'd say, oh, they're a blue shop and they have everything IBM from top to bottom, but that's not the case anymore.
 
Jared: You described the switch of being asked why you were there and then you just make it pretty, to this movement of now having this very large design studio in Austin, and other studios that you're building around the world, and how many designers?
Adam: The goal is for us to have 1,200 designers on the books by the end of 2016 now, and we're almost there, as it stands right now.

One of the first things that we had to do when we started the program was, we had to figure out who was calling themselves a designer in the company, and we reached out to every single one of them personally, and evaluated whether or not they were actually doing design work. After we normalized that, we understood, here are the number of existing designers in the company, and now here's how many we have to hire to get to that 1,200 number.
Karen: How do you manage the hiring process at that scale? How do you screen people? Honestly, Lou Adler said nothing about hiring 1,200 people.
Jared: That's a lot of proudest accomplishments.
Adam: [laughs] When he was saying, "Maybe you'll get three this year," I was like, oh, all right, different scale. In two weeks, we have a class of 75 designers all starting on the same day. The previous class of 68 is just finishing up their on-boarding process right now. How do we manage this? Currently, and these numbers are rough, but we've had over 10,000 applications from designers from all over. 
Karen: You just take the first 1,200?
Adam: Yeah. Well, if we did that, we would be done, but it's not about taking the first 1,200, obviously. We have a very robust recruiting and talent management group. Their sole focus is just building the pipeline and creating the demand and getting people to take their experiences that they've had with us at IBM Design, and bring them back to where they've been.

It's worth mentioning that, in stark contrast to a lot of what we've heard so far today, two-thirds of our hires are right out of school, so either right out of a graduate program or out of an undergraduate program. For a lot of them, we send them back to their schools to do the recruiting and to show the things that they're working on, the responsibilities that they're being given.

10,000 applications, 7,000 portfolio reviews, 5,000 phone screens, about 750 fly-downs, in which we take those people that make it through that first barrier, and we fly them to Austin for an intense, three-day interview, in which they're given exercises. They're asked to work with each other, so we can actually see how they behave working in a group and not just as a lone wolf. Then, from there, we've hired about 450 out of those 750.
Jared: How do you then integrate them with all the different product groups that are throughout Design? You've got all these folks coming in, and they're coming in through the Design Center, but eventually they've got to get out into product and service groups, right? 
Adam: Yeah. We have a Design boot-camp program, where it would be thoroughly unfair to take a group of people and just say, "Welcome to IBM. Now go work," because there's no discipline or rigor behind bringing the work and the profession of design to the organization. 

We created a Design boot-camp program, which I was referring to earlier, in which, if you come directly out of school, you start with a cohort of between 45 and 75 people.

It's a three-month program where they come to Austin. They have their own space. They work with one another. For the first six weeks, they're taught basics, like, "Here's how you work within the company. Here's how the company works," which is fairly large as it is.

Then, they're also taught principles around inclusive design, so bringing accessibility to the forefront instead of tacking it on at the end, learning basics around front-end development and being able to code within the JavaScript framework, and some basic stuff like that, with small micro-projects sprinkled throughout to apply that training.

In the second six weeks, they're actually given real projects that are sponsored by the business, and these projects that they work on, and the domain associated with it, are where they will end up being deployed. Then, after their three months, they're deployed out into the various business units.

For professional hires, they have a bit more of an accelerated time line, where they're brought in, they get three days of a Design camp, and then they're placed on their team after that.
Karen: Can you say a little bit more about these boot camps, and how you get people up to speed with the domain knowledge? If you're taking people right out of undergrad or at school, they, I'm quite certain, walk in without the level of insight into security or cognitive computing that you need them to have. How do you train them?
Adam: Sure. Part of that is done through that second six weeks in their Design camp experience. We also recognize that if they've just spent somewhere between four and eight years becoming the best designer they can be, they're not going to have that deep domain expertise in something that we specialize in, like security or cognitive or social or something like that.

Part of it is that we work with the receiving team that takes on the designers, to help them create a program that helps to on-board the designers, and give them an overview of what they're going to be working on, because even cognitive computing is a little broad, but it is about them learning on the job.

What I've seen from my consulting background, as well as many others who are consultants here, we've done and seen so many different types of business, whether it's pharmaceuticals or financial services or whatever, and we seem to pick it up rather quickly. It's a little combination of each. It's give them a running start, give them a program that gives them the basics when they get there, and then let them learn as they go.
Jared: Some of these folks are going off and working on the Watson team. Do you make them watch "Jeopardy" episodes?
Adam: [laughs] No, I don't make them do anything, actually.
Jared: [laughs]
Adam: Everyone says, "Your designers," "Your designers." They're actually not my designers. I have a team of about 15 or 20 people that are responsible for everything that you would see publicly around IBM Design, but everyone else goes to their business unit. When they arrive at the business unit, the business units, at the product level, will have a Design team lead, and at the portfolio level, there will be a Design executive that oversees the Design team leads.

They report in through those Design team leads and those Design executives, and we provide support to everybody in the mix. It's about being the best designer that you can be, not about directing them on their product work.
Jared: To follow the tradition here, what's something that you're particularly proud of about the Design camp?
Adam: For one, it's Bluemix, which is our platform as a service. That came out of the very first design camp, and the design really defined what Bluemix is as a product. Arguably, it's as important as Watson is to IBM.
Jared: Say a little bit more about what Bluemix is, for people that don't...
Adam: Bluemix as a platform, as a service, is, imagine that instead of having to go with the monolithic install of a bunch of IBM software on-premise, you can pluck things off the shelf, and use them and pay for them as you go, and by plugging them into whatever you've been using. So, service-ready is really what they are.
Jared: So if you need to check out things, there's a checkout component, if you need a ratings thing, there's a rating's component.
Adam: Exactly, and then there's complicated stuff like secure authentication, and things that you don't necessarily want to try to write yourself, or reinvent the wheel around.
Jared: Really? Everybody seems to write their own authentication apps. Madison did that that.
 [laughter]
Adam: We know how that worked out. In coming up with the design of the product itself, it really defined the product. I think a lot of times, what we struggle with as a group is, getting people over the hump that design is not like Photoshop color choices, and typography. 

That it is the purpose or intent behind what it is that's being planned. The way that you put it is, design is the rendering of intent. The way that I would say it is, it's the purpose planning and intent behind an action, factor, or object. It's not just about what just gets put up on the glass. It's about why this thing exists, and how it exists.
Jared: A few months ago, you graciously invited me to come by the studio, and you showed me the layout of the space. Can you say a little bit about the design here, because it's pretty awesome.
Adam: Super proud of it. It's 50,000 square feet across two floors. We have over 300 designers working in it full-time. When we were building the first floor, the eighth floor, we were stuck. There were 75 of us crammed into the most basic hallway, with four rooms that came off of it.

We broke the first design camp up into groups of 15. One day, one of the designers came out and said, "Hey, do you have a toolbox?" I said, "Yes, somewhere around here." I didn't really think about what I was doing when I handed it over.

A half hour later, I came back, and they had unscrewed all the furniture, this big heavy boardroom furniture, and they had rearranged the entire room to fit their purposes.

It became this cultural touchstone where, all of the teams would just pick up all their materials, and move it around, and shift it, based on what they were working on, or who they wanted to be working with. Around this time, we were doing the demolition of the eighth floor, and I made a call that all the furniture needed to be on wheels.

Actually, everything needed to be on wheels. If it wasn't on wheels, it needed to be light enough so that two people who were 5 feet tall, 90 pounds, could move it on their own, without help.

Everything on the floor is completely reconfigurable. That's the picture of one of the security work space on the eighth floor. That space changes on a regular basis. They all do. What we learned, what we were looking at before with the hanging light boards...
Jared: Can we roll back one slide? Yeah that.
Adam: Phil and I had both visited the Stanford D School, independently. We both came back saying the same things like, "Did you see that hanging white board system that they had?" If you haven't been there, they have these rails drilled into these huge oak beams. 

These white boards, you can take them off and put them on, and you can subdivide the spaces as you need. There are hooks for these personal light boards that go on there. We both fell in love with them, and we found out that because IDEO, Steelcase, and the D School are so tied to one another, that Steelcase had actually come up with this idea in the first place.

We called up Steelcase. They did this. This is the first of its kind for us, so we worked with Steelcase in the prototyping. The entire seventh floor is based on this model, where there are four large quadrants that can be split into four smaller quadrants, or split in half.

We also have the ability to pull those white boards down, and use them as table tops, so that you can write on top of them. Really, what this space is all about is being non-precious. It's not about being neat and clean. What we say is, "You better worry about what you drop, than what you drop it on."

There's sharpies, and post-its, and X-Acto blades, and stuff all over the place. It's about people doing their work and getting their hands dirty. 
Jared: Those hooks, the black dots on the boards, those are hooks for these personal white boards, and there are hundreds of those.
Adam: Hundreds.
Jared: People sort of bring them into a meeting, and they mark them up, and then they take them back to their desk, and they work with them?
Adam: Yeah. We also have them all stored on these like, Home Depot hand carts, so that the whole point behind this is...
Jared: ...You raided a Home Depot.
Adam: [laughs] We actually paid for them.
Jared: It's like throwing out a trash can. You go into a Home Depot and say, "I want three of the carts and..."
Adam: And just load them up.
 [laughter]
Adam: Shoot, I should have thought of that, but with the hand carts, the idea behind these spaces are the rapid construction and destruction of space. What we found is that, people work much differently when they're standing up, vertical, and next to one another.

When the day is done, and it's time for you to clear up, we've made it so that all of these components are removable. They can put them back on the cart, and basically tape them together, and put a post-it on there that says, "This is Adam's set of whiteboards. Do not erase."

Then if you want, you can come back, and set it up in one of the other spaces, or the same space. You can use all of the material all over again, because what we really determined was, the more we work on whiteboards and vertically, this is the externalization of our minds.

This is our collective brainpower that's being spewed out onto the walls. Often times we get spatial with this that. In a space like this, an idea will get thrown up on the wall, and you'll do a whole mess of drawings around it, and you sort of remember, "Oh, it's in that lower back, right corner, over there." People have the tendency to go back to the physical space, to remind themselves of what they've worked on.
Jared: "Spewed out onto the walls," is my new favorite band name.
 [laughter]
Adam: There you go. I managed to make the list.
 [laughter]
Karen: I would love to ask you some of the structural questions around where the designers sit, and report into, in the organization. Some of them are reporting through the design studio, but many more of them are actually distributed out through the individual business units.

How do you ensure that they have the right level, title, career path? How do you ensure that designers, or performance, is being reviewed and evaluated appropriately?
Adam: Similar to what Samantha was talking about with GE, we have a set of values and what we call personal business commitments to IBM. We're all evaluated against the personal business commitments. Some of them come from Ginni, at the CEO level.

Some of them come from down below, at the Senior Vice-President level, but we set the PBCs for all the designers, and they're sent out to anybody who sits...
Jared: PBCs are...?
 
Adam: ...Personal business commitments, sorry. I work at IBM. Everything gets crunched down into three letters.
 
[laughter] 
Adam: We write those personal business commitments for the designers, and then they're distributed. One of the things that makes this quite easy to do is that, my wife Jodie, for the first two years of us doing IBM design, she wrestled the HR system to the ground, and made it so that we have an official job title and job code in the HR system, with an associated career path.

For the first time ever, designers have a seat at the table, along with engineers. They have the influence over the way that the business is run, whether it's at the portfolio business unit, or company level, and that they have a path that they can look at to become an IBM fellow, just the same way that an engineer is. This is the first time that this has ever been.

This allows us to maintain equal pay, and then because everybody is in that job code, we actually know who is a designer, so that we can line up all of their evaluation criteria together.
 
Jared: This is a big deal, right?
 
Adam: Huge.
 
Karen: Huge deal.
 
Adam: Yeah, a huge deal. Yeah, absolutely.
 
Jared: Because years ago I remember working with designers at IBM, and the designers were sort of lost in the HR system.
 
Adam: When I was hired I was an IT specialist, if you can figure that out. It didn't make any sense, and I don't know how many different phone-calls and emails I had to send just to get switched to consultant, and they grudgingly did that after 10 years. Now that we can all be slotted into a designer job code makes life a lot easier for a number of reasons.
 
Karen: How's that effect performance evaluations that are being done? Not by you, but by people in all the various other business units?
 
Adam: Again, it's through those personal business commitments that these are the criteria that we're all evaluated on. As a designer they're being evaluated on the contribution that design makes to the business.

We're brought in as consultants sometimes to the business units, if they have a harder time making a determination between things, but for the most part they have the framework in place already, and the designers that are there are the ones making the judgment calls.
 
Jared: How did you get executive buy-in to do all of this?
 
Adam: There's a long story, and the shortened version of it is that my manager who is the general manager of design, Phil Gilbert is, prior to the formation of IBM design, was talking with his manager and they were joking and the said, "So what's next for you Phil?" And Phil sort of said jokingly, "Well if it were up to me I would redesign every single last piece of software that IBM made."

It became one of those "Be careful what you wish for" moments. After a few discussions and some planning with Jenny herself, Phil is dotted line to Jenny.
 
Jared: He's the general manager of design for IBM?
 
Adam: That's correct. Across the whole company. We have a charter from her, the funding from her, so we explained and justified what we needed to do in order to do this right, and not have it be a halfhearted attempt that falls flat six months down the road.
 
Jared: When you say you have 1200 people, and you're dotted line to Jenny, do they ever see her?
 
Jared: Yeah. When we opened the studio, she brought the entire senior leadership team to Austin and spent half a day with us and sat with the designers, made herself available, and then she came back maybe two or three months ago, where she interviewed Randall Stevenson the CEO of AT&T in the studio.

Did a 25 minute Q&A with all the designers in the studio, and is keenly aware of what we're doing. It's not like we're something that's sort of off to the side. She's directly involved, not all the time, but she does come by.
 
Jared: When she brought the senior staff, explain that thing? There's a noun in there, but I don't want to.
 
Adam: The thing. [laughs] Because we're designers we can't do anything straightforward. When we opened the studio, rather than simply have her cut a ribbon, which is sort of cliché, I don't know, probably over 500 discarded post-it notes from various sessions and arranged them on this giant board.

Then because these designers were particularly masochistic, they took these bigger post-it notes and they projected each of the senior leadership team's names, individual names, in "Helvetica" up onto these things and then hand-drew their names in Helvetica so that it was brand appropriate and all that.

Instead of having them cut the ribbon, they all autographed their own post-it note. Each post-it note was based on one of our company values. We had that framed in this massive glass frame thing.
 
Jared: Did senior leadership appreciate that they had traced Helvetica and not "Arial?"
 
Adam: No. No, they didn't know. They were just like, "Oh, that's neat."
 
Jared: We have some great questions that have come in...
 
Karen: Should we ask? Let's see, Michael. Michael, you know your last name.
 
Jared: You can tell us your last name, so that everybody knows we're avoiding pronouncing it.
 
Karen: Right there? So you ask a couple of questions. One about how they get deployed into the business units, and then what sort of support gets provided after?
 
Michael: It's kind of a follow up to a question that I want to ask, what actually happens to these designers once they leave boot-camp, you mentioned that they get deployed. I assume that means across the country, or across the globe maybe.

The follow-up to that is, once they are deployed in these remote places, what kind of tools do they have to phone home to the design group to discover what the other business units are doing. Are there any tools for collaboration once they're in these separate places, things like that, and do they ever see each other again like do you have [inaudible 23:55] .
 
[laughter] 
Adam: All right, three part question. I hope you were keeping track. The first thing is the reason for the creation of the studios was to house designers together. Most of the people that we hire, they actually, when they get deployed, they simply go upstairs to the eighth floor and join their team up there.

Some people will go to Astor Place in New York, some will go to Hursley, England, or Dublin, Ireland, or Shanghai. For the most part they're sitting with other designers, and partially this is for a couple reasons. One is, as we were starting the program, engineers didn't know what to do with a new set of designers that they've never worked with before. 

Sometimes that was like grinding gears a little bit. By co-locating them with other designers, they're able to support one another sitting near one another. Some do end up going remote, for the most part, though, they're in a studio. However, their teams are spread out all over the world. 

There's 388,000 people in IBM in 171 countries. Most of their development teams are in at least two or three different time zones, and probably four or five countries.
 
Jared: IBM already has a culture of working remotely?
 
Adam: Absolutely.
 
Jared: This is just utilizing that existing code. You didn't have to do something new for designers to work remotely with their team?
 
Adam: No, in fact, some of this stuff from a collaboration perspective, after we were doing some of the design camps for the product teams when they would fly down, they would say, "This is great, but how do we do these post-it note things when we sit in three different places?"

We formed a partnership with Merrily, which is a cloud based sticky note tool, which everybody loves and we use all the time. The teams all use that together. Engineers, project managers, and designers, regardless of where they sit.

We use Slack an awful lot, which is why you've seen me pretty active on Slack today, it's just constantly going in the background for all of us. There's also been some pretty interesting homegrown solutions. I had one team that they lost one of their designers, they had to go to North Carolina while they were selling their house.

They were sitting at RTP in the Research Triangle Park location, but they missed the face-to-face interaction, so they went on eBay and they bought a used IV pole, and then they bought a little clamp that they stuck an iPad in, and they turned her on, on FaceTime, and left her on all day. When she was sitting at the desk with everybody else, they lowered the pole so she was at head height.

They said. "Hey Erin, do you have that...?" "Oh yeah," and she would just do a file transfer instead of handing something over. When she wanted to talk to somebody else in the studio, they would just raise her up to standing height and wheel her around.

She came back to visit, and the same thing, we all laughed the first time we saw it, and it was kind of ridiculous, and then she came back and she said, "Well I have to personalize this," so she left one of her shawls and tied it around the iPad. Then other teams just started doing this on their own.

There was one team that had a Canadian product manager, so he brought down this big tube with a big pompom and a maple leaf on it and he stuck that on top of it, so you could tell whose iPod was who.

You would have these design competency meetings or town halls, and you'd look across a crowd as big as this if not bigger, and then you'd see there's a person on an iPad stick, and there's a person on an iPad stick. It got to the point where there were no longer giggles, it was just, "Oh, there's Jared and Karen, but they're not there."

 [laughter]

We've done a lot of homegrown stuff with that too. There was a third part to the question, and I can't remember what it was.
 
Jared: It's "Do the teams have a chance to see each other after," but since they're co-located...
 
Adam: Yeah, for the most part they are co-located, and then they do come back. We also have our designers go into other studios as well, so they do get to see each other.
 
Karen: Let me ask a question on somebody's behalf. When you are doing performance evaluations or setting up a framework for performance evaluations, do you have a way of measuring the contribution the design, or an individual designer makes to the business, and can you separate that out from...
 
Adam: It's very difficult. I think something that needs to be said in general is, at least from my point of view is, we need to let go of ownership of design. If design is just the rendering of intent, we choose to work in certain media. Whether it's a wire-frame or a set of requirements, but code is just as important. I don't know how you would split apart like, "Oh, the reason why this thing succeeded was because of the design."

I think if there was a year over year release upgrade to some product, and that the only thing that was done for the next release was something that was user centered and more about the user interface, or the user experience and sales went up by 20 percent, or whatever it is, I think you can point to it that way, but it's very difficult to split out, this was designs contributions.

It's more about "Did they work together as a team?" As soon as we start saying, "Design's over here, engineering's over here, product managing's over here," we run into the same problem that we've all been talking about to one degree or another, which is like "How do you deal with them? Who's your favorite?" and "Why do we butt heads?"

It's because we're not thinking about approaching this as a team, we're thinking about coming at it from our own disciplines, and then we're sort of forced together as a team.
 
Jared: This idea of working with the engineers, and sort of changing that part of the culture, and we've talked a lot about culture over the last two days. IBM being more than a hundred years old and having a deep history and being very engineering driven for much of that, but also having the marketing ethos of the guys in the blue suits and the white shirts.

There's always been a very strong culture at IBM, but design's contribution to that culture has always been sort of backstage, and how've you change culture in particular with putting all these designers into the world and having them work with these teams? As part of the boot-camp are you training them to be cultured change agents in the regard? Does that happen independently?
 
Adam: It's a little of both. I wouldn't say there's a deliberate, like we're training you to be a culture change agent, but we are training them on how to behave, that people want to be treated a certain way. If designers come in saying, "Oh, everybody, you've sucked for so long and we're here to save the day," that doesn't really go over very well.

There's a part of it which is especially if they're coming right out of school. You've never really worked with these individuals before, so you what their expectations are versus what your expectations are may be here and here, and this is why I thought, yesterday Karen just hit the center of the bulls-eye.

It's all about alignment. Internally it is all about getting our efforts aligned properly, and understanding that we all are trying to do the same thing. Nobody shows up to work saying, "I'm going to make a really bad decision today." Everybody thinks they're making the right call.

We talk so much about building and gaining empathy for our users, and yet designers are the worst when it comes to using our own tools to solve our own problems. What about empathy for engineers and what they go through, or product managers? A lot of times it's just about having them use the tools that they already have and applying them inwardly.

Culture-wise, I think at the beginning there was a lot that had to do with very high touch with us, and the folks in the core IBM design team really working closely together with these teams to show the value of what we were doing.

Now it's all shifted to "How do we do this at scale? How do we do this when we're not in the room?" It can't be just "Here's a video that we recorded," and watch this video. It has to be that we're passing it down to the practitioner, so by putting the same capabilities into the hands of the design executives, and the design team leads. Their responsibility is to then take these ideas and spread them further.
 
Karen: I think one of the themes that's come up in a number of our interviews is, design moving from the hit it with the pretty stick, to actually influencing the product road map, or the vision, or the direction, of the product. To moving into a more strategic role, actually influencing the future direction of the company. Can you talk about how that's happening at IBM? How your team's doing that?
 
Adam: Yes. The very first thing we did was that we developed and introduced IBM Design Thinking. The reason why the IBM is there, is not to necessarily brand it as our own, but we sat down with David Kelley, and with Tim Brown. We talked to them about what they were going through when they were developing the concepts of Design Thinking. The one thing that they both said was, "We really blew it when we called this thing Design Thinking."

 [laughter]

The second was, "It works great when you're doing it in these small workshops at Stanford, but it starts to break apart and fail at scale," and so what we did was, we took the core tenets of Design Thinking. Exploration, understanding, ideating, making, and evaluating.

We took those, and then we added three governing constructs on top. The first being a sponsor-user, not just going out and doing standard engagements with users, but signing up users from paying clients already, and people who are using them. Getting them to be co-designing with us, from somewhere between 10 and 50 hours, over the course of a release. 

The second thing is a construct called The Playback, which is very similar to what a lot of us would call a critique. It is these open dialogues where just the design work is shown at a high frame rate. The whole thing maybe over and done within 10 minutes, to show the actual work. 

It maybe that they're just phone snaps of scribbles on a whiteboard for the first one. Senior executives are invited to every last one of them. Everybody attends these things. They don't always attend every last one of them, but they do attend them.

It gives everybody a chance to look at what's being designed and say, "I think we lost the plot." "I think we were right on track," and keep pushing forward. Or, "Where are we going to go next after this is done?" It gives us a series of checkpoints that allows us to stay honest with each other throughout, so there's no big unveiling, and everyone goes, "That's not what we asked for."

The last one, which is probably the most complicated to understand, or to explain is, this concept of hills. That's a military term taken from commanders' intent which is, you're going to take the hill. You can train for a mission all you want, but inevitably, things go sideways when the mission starts, but what doesn't change is that you take the hill. 

The hills are three, and only three articulations of what you're going to provide in any given release, so that we don't bite off more than we can chew, so that we can keep the team's focus on what's necessary to have a successful release. It's always user-focused. This gives teams a way to focus the effort on a per release basis. Those three things. That's a long-winded answer to your question, but that is part of how we're bringing this out institutionally.
 
Jared: What's the mechanism for getting it out to the rest of IBM, because there's a lot of people who need to know this stuff. What sort of machinery do you have to get them to be a part of that program?
 
Adam: It's surprisingly simple. There's a pier to the new higher design camp, which is the design camp for product teams that lasts a week. Per team, I think we have like five or six different product teams that attend. They all fly to Austin, they spend a week there, and its two engineers, two product managers, and two designers from the team. They're taught and then they're given all the stuff that they need to bring that back to their larger team.
 
Jared: It's all Train The Trainer effort?
 
Adam: Yes, and there's a formal Train The Trainer effort going on within GBS, so that the GBS designers, the Global Business Services, the consultants, that they're being trained on how to use IBM design thinking for client engagements, which requires a few nuanced twists to how we do the work.

Simply enough, we have an internal website that we put all the material up on, that allows people to come and get it. My team is currently working on the first public version of this, which should be out sometime in the first quarter, in which we will share this with everybody, the way that we've done with IBM Design Language as well.
 
Karen: You are our last interview of the event. What advice do you have for other organizations, other people who are looking to bring about this kind of transformation, or use design as a competitive advantage?
 
Adam: Let's see if I can remember. One is, and I'm going off of what people asking in the Slack Channel to. The space that you sit in, is what you make of it. For everything that we have in our studio, there is something that is home-made, that does the same job. We've made up 75 whiteboards with Xerox, that you can find the directions in the D School website.

It is about providing your team what they need to feel, that they are prepared and they have what they need. It's not about what you're giving corporately; it's about you making the space what you need. Ask for forgiveness, not permission, as far as that stuff goes.

I already said, let go of the ownership, share design. It's more fun this way when you actually share design with everybody, as opposed to trying to claim it for yourself. What else? Based on what I was saying about Karen's statement yesterday, about she's in the deliverable business, but she's in the outcome business.

Artifacts are not sacred, don't get into holy wars over which artifact, or which method, or which tool you use is better than another. It is about the outcome, plan for your outcomes. That's the most important thing. Get there how you have to.

Wherever possible, I understand that sometimes it is a competitive advantage, but when you come up with something that advances the state of design for your own team, or for yourself, if you have the option, share it freely with as many people as possible.

We all get smarter and better because of this. That's part of the reason why we show up at an event like this, is to hear the stories that everybody else has to tell. When we're networking outside, to be able to listen to how people address certain problems.

If you come up with something new, share it. There's certainly no need to go by the book all the time. If you invent something new, heck how many things did you invent from the clear blue when it came to usability testing, over the course of years...
 
Jared: We were making all sorts of crap up.
 
[laughter] 
Adam: Still making crap up to day, so if you make something up, share it. It doesn't mean that it's right or wrong, because somebody else has written a book, or a blog post, or something else about it, but just share.
 
Karen: That went well. Thank you so much. This has been a fantastic ending to our conversation.
 
Adam: My pleasure, thanks for having me.
 
[applause] 

UX Advantage: Reinventing the PayPal.com Experience

Bill Scott - Bill describes what it took to win buy-in from executives, bolster communication, and push boundaries as he led a remarkable transformation in Paypal’s corporate structure.

Jared Spool: ...Bill is VP of Next Gen...

Bill Scott: Commerce.

Jared: ...Commerce, at PayPal. I've known him since the early days, back when he was at Sabre.

Bill: We go back about 40 years, or so, ain't it something like that?

Jared: Yeah. Something like that.

Bill: 50, 60, something.

Jared: Something incredible.

Bill: He was a crib mate [laughs] .

Jared: Yeah. You've been at PayPal now...

Bill: Almost four years.

Jared: Four years. Since you've been there, you've had proud of?

Bill: I think when we reinvented Checkout. Checkout product had pretty much stayed the same, for probably five years. When David Marcus became president of PayPal, he had been part of an acquisition there.

Jared: Checkout is the process where a third party on an e-commerce site...

Bill: Yeah. You're on an e-commerce site.

Jared: ...has an option that says, "Pay with PayPal."

Bill: Pay with PayPal, and you go through the flow. The trick with Checkout is that if you just have a slight variance in conversion, of the amount of people who were successful going through a payment checkout flow, it's a lot of dollars. To change that is not a trivial thing.

When David Marcus, who had been part of the acquisition, took over as president, he had a startup DNA. He got myself, and few other folks together in a room and said, "Let's rethink Checkout from scratch." The way we did it was we got maybe two designers, three designers, two or three product folks, three engineers, and we got a war room, a conference room.

We holed up in there, and basically started sketching. Within a few days we had a working prototype, that took David's Balsamiq Mockup. I love when a president of Accounting does Balsamiq Mockups. Took it and we made it live, to actually process some payments.

By the next week we had users coming in, and doing usability testing. That was a really different way of working, and I think that's one of them.

Jared: Help us understand how different that was, for PayPal.

Bill: Basically Jeff Gothelf, you guys may know Jeff

Gothelf, who does the LEAN UX book and workshops. We had him coming in, and do a forensics on the design organization. One of my favorite conversations with Jeff was interviewing a designer, who said they had spent three weeks creating these beautiful Photoshop mockups.

They went to show it to the engineer, and the engineering team laughed at it. The designer went away crying. I call it pissing on your Picasso. The whole mode of operation was to create something, throw it over the wall. Karen was my partner in crime there, before she went to MasterCard a year and a half ago, at PayPal.

She's nodding her head, and she knows the experience.

Jared: Karen Pascoe.

Bill: Karen Pascoe. Yeah. That was the difference. I remember Cody, who was our designer on that Checkout product in that room, made an interesting observation the first day. We'd been sketching on the whiteboard, and one of my guys was coding it in real time.

At the end he goes, "OK. Should I use an OmniGraffle, should I use Photoshop, should I use Ajera? Where should I put this in?" Jeff, who had written the code, turns the laptop around and says, "Well here it is." Cody goes, "Well what's my job?" He had been seduced into thinking his job was just documentation.

Really what our job is, is delivering experiences together. That was the big "Aha" moment, in that room.

Karen McGrane: I'm so impressed, by everything you've been able to accomplish at PayPal and beyond. What advice would you have for other organizations, that are looking to take on this transformation? What principles, do you keep in mind?

Bill: I think to change an organization that big, you have to have support. There's no doubt about that. You also have to get a groundswell in the grassroots. To do that, you got to yourself have some very strong principles you believe in. What's going to happen is it takes an enormous amount of persistence, beyond measure, to continue to go against all the things that will fight you in an organization.

Basically organizations are set up to stay the same, or go the wrong way. If there's anybody who said, build up an organization and resist any change. You have to have this persistence, that is held in some strongly held beliefs. Now these principles, these beliefs, don't change from company to company.

Everywhere you go, these same principles hold. Some of mine for teams are getting to a shared understanding, between all the disciplines. Engineering, design, product, marketing, legal. Everybody has a shared understanding of how we get product out the door, and deep collaboration.

We know how to work together. We believe that it's a team sport, that we work together, and then that continuous customer feedback. Believing that if we can get back to the problem, we'll stop defending the solution, and we'll start coming up with great ideas.

Those are three key principles for me, the understanding, the collaboration, and the customer feedback, that I used over and over in my talking to teams. When you have that persistence, that gives you something that just roots you and holds you solid, but then you have to have improv. You can't just be rigid.

I've seen people come into organizations, who try to change an organization, and they have a playbook they did at the last company. You've probably all seen that, right? They come in and they've got some processes, or they got some programs. Whatever it is, they bring it, and they try to apply it.

It's met with such resistance, it's so ugly, it's so stale. When I came in, basically my pitch to be hired was, "I don't know what the hell I'm going to do, but this is what I believe. I'll make sure that these beliefs, get in the organization." They were dumb enough to buy into that pitch.

[laughter]

Bill: I basically was able to go in and have humility to start with because I felt like within a company like that, even if on the outside you go, "Man, this is so screwed up," because PayPal in 2011, the user experience looks like it was from 1999. It was so horrible, you go, "Is there anybody here any good?"

Of course there are lots of good people in that organization, but the problem is everybody gets stuck, so you have to believe there's wisdom in the crowds. You have to believe that there are a lot of people who want to do the right thing, so you can find partners, allies, and you can start teaming with them.

That combination of that dogged persistence with a humility and improv to listen and to reflect, and then to put those together into a playbook, a solution that's in context for the organization. Listen, that changes too. That's one of the shocking things. You sort of get something going, and and you go, "Wow, that's going great."

Then you have the leader of organization leave the company, or a big reorg happens, new people come in, other people leave. All of the sudden, the playbook doesn't quite work the same. You've got to readjust and do it again. That's where the persistence really gets really hard, because you've already done this once. Now it's twice. Now it's three times. It's just that constant persistence and improv.

Jared: Are there principles that drive that playbook? Do you use some sort of overarching principles of what you guys were about and what you're trying to do that lay out the pieces of the playbook, so that when that shift happens, the playbook ends up changing?

Bill: To me, that third principle I talked about, that customer feedback, is the most important, because every time I see teams that get stuck or start fighting, or you got design team versus the product team versus engineering team, they've always forgotten the problem. That's my experience, anyway. Maybe I'm too simplistic and reduce it too much, but it does seem to be the case.

The first thing I try to do is start asking what problem we're trying to solve, and then make sure you get customers in front of our people. Have them go to usability, have them do deep design research, actually do field visits, follow the customer. Any of those things we can do to get them to see a real user using the product.

It's amazing how it jars everybody out of their comfort zone. The designer starts thinking, "How could I solve this problem?" The engineer starts thinking, "I think I could do this with the code that..."

If you get legal, you have the "yes" lawyers, that's what we experience too. If we got legal in the room with us with a whiteboard, and say, "Here's the problem we're trying to solve. You have problems you're trying to solve too. Walk us through that. We'll walk you through ours." Then it was amazing how the legal team became a design team.

Jared: What happens when you put a lawyer in front of a whiteboard? Do they just dissolve?

[laughter]

Bill: These didn't anyway. I think that could be the case. That's a really good point.

Jared: "It's a document that gets erased! We can't have this!"

[laughter]

Bill: They used the Sharpie.

[laughter]

Jared: That'll do it. They know how to leave their mark on an organization.

Bill: It's pretty simplistic but I keep putting in my head that if I can put a smart team together and I can soak them with enough context of the user's problems, good things will happen.

Karen: I like how you described that, like soaking them in customer feedback.

Bill: In fact, the team I have now went from having a really large organization, 400 or 500 people to just 10 people. They're design, product engineering, analytics, strategy in one team.

My job at this point is, how can I get as much context about the payments industry, about the financial industries, FinTech, everything in to their heads, expose them to as many problems as possible, because this team is really good and they will come up with great ideas that I never conceived of.

Jared: You've been at PayPal now for four years. When I first met you, you had just come to Yahoo, and then you went to Netflix, and then you went someplace else, and then you went back to Netflix.

Bill: Couldn't get enough.

Jared: You could not get enough of Netflix.

Bill: Yeah, I've finally got enough the second time.

[laughter]

Jared: You've finally got enough. You were at Sabre before Yahoo. These are structurally very different organizations.

Are there things that that perspective gives you of these different organizations, where you realize there are just certain things about how do you start working with a new culture, how do you get in there, do you have to tweak that playbook that you have, that you're using based on the culture? Are there just things that basically just work and you just bring that with you?

Bill: You don't do that. The first thing to recognize is that what's common with all these organizations is there are people there. We're all kind of messed up.

Jared: It's like Soylent Green, right? They're made of people.

[laughter]

Bill: Exactly. I don't know if you've noticed, but we're all pretty messed up in different ways. When you put us all together, it's a pretty interesting soup that happens. Each of us have an agenda. We all try to have pure motives and pure agenda, but we all have agenda. We all have some self-interest, and everything else.

Recognizing that out of the gate, and then as you start working with people, trying to understand where they're coming from, and how can I help make them successful, without sacrificing your principles, without sacrificing the product.

Sometimes it's as simple as, you find a person who's a decision-maker, and you just take more of a back seat, a supporting role, making that person really successful and you just don't care about your credit, right?

I have a Zen view of it, that if you end up doing the right thing over time, the organization recognizes it. If they don't, you shouldn't be there. It's pretty simple. It gives you some freedom to not worry about who's getting the credit, and just go do the right thing. Then, you do have to, in these big organizations, in every case - PayPal probably the more extreme case. I had to note certain people in the organization that I could not move.

They just weren't going to change, so I had to route around them. I tried not to do too much of the routing around, but some people you just had to route around. You just prayed for the day they left. I'm just being real honest with you here, I remember driving into work one day and we had to call this one person that had left the organization, they were telling us.

I was in the car. The 101, I'm sure people are looking at me crazy, I'm going "Yes! Yes! Yes! That [indecipherable 12:33] is gone."

[laughter]

Bill: They caused me so much grief, it is very frustrating when you're working with people who are very stubborn. I had a guy sit down with me and told me he was upset with me. I said why. He said, "You're affecting my goals. I'm not able to achieve my stated goals." I said, "I didn't come here, actually, to help you achieve your goals." Just, newsflash.

That was the kind of people sometimes I had to work with, and Karen's nodding her head. She knows. She and I could name the same the people.

Karen: Let me follow up real quick, and ask, similarly, how did you get executive air cover in all of these different roles. You worked with Larry Tesler at Yahoo, and Reed Hastings at Netflix. How do you get executives to support you?

Bill: I think, speaking the language of the customer is actually speaking the language of business, if you think about it. When I came into PayPal, for example, I didn't come in and say, "Hey," because one of the big things I did was change the whole front-end technology stack, so we could get to continuous delivery and fast releases.

I didn't come in and say, "You know what, we need to stop doing Java, we need to start doing No JS," which is what we did. I didn't know if that's really what I wanted to do or not. I came and said, "You know what we need to do, we need to get to rapid experimentation." The UI layer is the experimentation layer.

What I would do, every time I've gotten a chance to be with one of our executives I'd already thought of Tweetable Moments. Tweets that would summarize something. I would actually give that sound bite to them. When you think about it, the user interface layer, is the experimentation layer. Then, I would hear other people saying that, which was cool.

I put these little memes out there. "The hardest thing that we face is moving from a culture of delivery to a culture of learning." They'd go, "Yeah. That's right." Then, we'd talk about that, right? It would start a conversation. To me, these Tweetable moments were key to that, because you only get a few moments of an exec's time. You want to make it count, and it should be something that has a business impact.

I tried to be, you know I was coming from an engineering perspective in the engineering teams. I try not to wear one hat. I try to wear the design hat, the product hat, the legal hat, the business hat, so that when I talk to anybody, I can come from their angle and connect with them. I think that connecting at the base need level, what's the real need we're trying to solve, resonates.

If you come to leadership you're saying, "You know, we need to go to No JS," Who cares. That's great, but three years from now, it'll be something different. Those aren't really the solutions. How are we going to get to a place where we can beat our competition. We can deliver better products to our customers. We can do it really fast. We can learn, learn, learn, learn, learn.

The learning meme or mantra really resonates with leadership.

Jared: The structure of PayPal. Checkout is just one part of the PayPal empire. When you joined it, PayPal was part of a bigger eBay empire, which it's now severed itself from in some way. At the other organizations that you worked at, did you see that there was organizational structure influencing the way decisions were being made, and how did you just break down those?

Bill: It's really huge. I think one of the things I learned at PayPal being in executive leadership, was just how important in a reorganization, who you end up being in staff with each week affects the whole organization. I don't think you realize it until you're actually in senior leadership and you start having that weekly staff meeting.

Now, professional services is on the same staff with you. Guess what. Professional services issues will come more to your fore, and you'll start solving those problems that are affecting. Your merchants out in the field with trying to use the product, or developers using the product.

Whereas from there, in a different organization, you never are exposed to them except maybe once a quarter, you just as a human don't think as much about it. Here, you're in a staff meeting, it makes a huge difference.

The real work that happens in an organization always happens across organizational boundaries. You have to recognize where those connections are coming from. You also then have to then go, "OK, well where are the connections not happening," Then you really reach across the aisle and start making this happen. When I first got there, I did 100 interviews in about 45 days, something like that, right?

Jared: Who were you interviewing?

Bill: Everybody I could in the company. Anybody in leadership, individual distributors, leaders, executives.

Jared: Across different groups.

Bill: I tried to make it as far as I could. I created a full organizational map of the company. What everybody did. I'd create charts of all that so I understood what everybody's role was, what the subtlety of the different roles were. In a big organization, there's all these weird overlaps. They're not quite overlaps, but they're not fully overlapped.

Understanding it functionally, how somebody is working, that political context, that organizational context was really key. Once you've had that conversation, you've connected with somebody, even though it was 30 minutes, you have the right to go back and ask more questions.

There's somebody you've connected with that they'll listen to you the next time, if you made a good impression in your conversation.

Then you learn a hell of a lot. You get this organizational context and you get an empathy for the organization. I call what I did there at PayPal really was supplying design research methods to changing an organization. The actual organization is my user, in one sense.

You're doing those hill studies. You're talking to people. You're observing how people work. You're starting to diagnose and do forensics, and then come up with maybe what a playbook would do.

Jared: If you guys are starting to form questions, and I hope you are, the session questions channel in SWAK is where to put them, and then we'll go to them in just a few minutes. In terms of all these interviews that you did, I'm very curious when you got there, what was the viewpoint of design and user experience and how did that set your agenda in terms of what you wanted to accomplish?

Bill: Both design and the front-end engineering which I later called user interface engineering...

[crosstalk]

Bill: ...were very disrespected. They were pretty low in the totem pole, especially the front-end engineers.

Jared: Why was it? Do you have a sense?

Bill: I know in the engineering side, I can comment on that. I can comment a little on the design side, too. On the engineering side, what had happened was you had developers they called devs and then you had Web dev. It was actually said like that Web dev. It went down. It was like [does inflection] Web dev. The dev was lower and the Web dev and then dev.

It was basically, they had this templating language you had to use. You didn't write any real code, you wrote templates. There was not much respect from the actual engineers for the Web devs. I said we're now user interface engineers. Those guys are dev but we're engineers. I just upped the game and brought in some really awesome people and we started solving really hard problems. Before long, we became the most respected engineering organization in the company. We changed that game.

On the design side, there had been so many different leaders that had come into PayPal earlier on that had their own issues. It was seen more as just the last the resort. There was this big planning process that would happen, and then at the very end, design would get a chance to put their stake in it.

What really changed it was when Hendrik Kleinsmiede came in to PayPal lead design, Karen knows real well. In fact, Hendrik is over at VISA now. Hendrik, really, was good friends with Jeff Gothelf and believed in UX and Lean Startup. He immediately, before he got on, I reached out to him and said, "I hope you are thinking like this. He said, "I am. Let's do it together."

Us together, having that combined mindset brought design to the fore because we were partnered with our president. Basically, it was me and Hendrik leading the charge for the new checkout product. That put the head of design and the head of our engineering together to lead that and we were successful. It didn't hurt to be successful. It definitely helps.

Karen: Can you talk a little bit about leading the charge for more Lean UX or more Agile methods. You got a couple of slides if you want us to bring those up.

Bill: Yeah, just flip real quick, one forward. That was the room we got into to create a project we called Hermes at that time to recreate checkout. There was just a few engineers, few designers, product folks, a lot of white boarding sketching, a lot of real-time coding.

That was the start-up. What happened was more and more teams saw this effect of this and definitely wanted to do this. If engineers are exposed to this and designers are exposed to this and product people are exposed to this, there are very few who don't want to do this.

Jared: Walk us through this a little.

Bill: Here, what we did was we were doing a lot of white boarding. You could see on the left at the bottom, some mockups, printups. Basically, Eric and Jeff were doing a lot of the coding real-time. We go to usability on the right-hand side. We do that every week. Our rhythm at that time was Tuesday and Wednesday were about building for the next usability session. Thursday was usability sessions.

We did it in a write format, rapid duration testing environment where you change the code through the day as you need to. The engineers, designers, product people were in the room together observing users on Thursday. This required our usability team to change the way they work because they had to bring a report the next day. Friday, they had a report for us. David, our president, would also drop in to listen.

We avoided a lot of the highest paid person in room phenomenon because we had that report. David would have ideas. He was also pretty careful about saying, "Don't just listen to me, let's do what the users want to do," which was still hard to do because you pretend to listen to the leader. We would just test it.

Then, we'd decide what we were going to do and then Monday, Tuesday, Wednesday, we'd do that. We did that probably for 12 to 16 weeks that rhythm then we slowed it down to every two weeks. From there, just...

Jared: Before this project, what was the rhythm like?

Bill: Probably like towards mid or end of a project that we'd go to usability to see what they built.

Jared: Which is how many weeks then?

Bill: Probably, it could be three months or something like that.

Jared: You went from these three to five months cycles down to one week.

Bill: Yeah. The interesting thing is, it took us a long time to actually roll this new checkout out because you're replacing an existing product. It required the whole bottom of technology, all of the payments core platform to be refactored and redone. It changed the whole organization which was hard.

It took a couple of years because you have to get your conversion rate, the amount of people that are successful going through checked out to be as good or better than the previous one. It didn't matter if it's ugly or not. It's working. We surpassed that.

Now, we're in almost full ramp mode. It took a while to get there. That 12 weeks or so that we did that, the design really hasn't changed much since that 12 weeks.

There are a lot of tweaks we've done. The basic framework we came up with, the way we handle the wallet, bang different funding instruments, handling things as diverse as credit or whatever else while you're checking out. All that came about during these usability studies. That was a pretty fruitful time.

Jared: Where's Eric Boston? He's here somewhere over there. Elaine can come over to you with a mic. Eric's got a question over here.

Bill: This is the stump the chump?

Jared: This is. This is. Who's our contestant playing for?

[laughter]

[crosstalk]

Jared: Over there?

[crosstalk]

Eric Boston: I didn't know I was going to actually ask the question. My question is more around our team. We have multiple UX teams throughout the organization because when you have focuses on mobile specifically, web specifically and then we have a bunch of internal teams that are doing stuff just for employee facing apps.

We've been changing through the pace of UX within the organization. We still have teams that have UX in the titles, they do stuff that seems UX-like but yet they're implementation seems like it's always more like, "Well, we believe this therefore that's true," versus going out to the customer, doing interviews, doing prototypes. It's around like, we have some buy-in, but then it's like there's a lot of different practices as far as how that buy-in is implemented. How do you get alignment on within those teams? That's what I guess I'm asking about.

Bill: The question really is about alignment among various UX teams. They don't report into the same organization. They're in separate ones. We tried a lot of different things like Yahoo. I did the Yahoo design pattern library to help coalesce some of that. Then I've seen other groups, I saw PayPal doing this too, where they were trying to create a pattern library and really it was a pattern police which never works.

I've been in central UED design organizations. I have some experience around this. What I've found, again, I'm going to go back to the same principle that I talked about earlier, is that if that central core UED team that is developing really the core styles, the guidelines, the patterns or whatever approach they're taking, in some sense, it almost doesn't matter what approach they're taking in. It depends on the organization.

If they're not solving the problems for those other groups helping them make them better, they're not going to be successful. What the classic mistake is, a centralized team gets really frustrated because this team's going off this way and that team is going off this way and this team is going off that way. If they actually think of them as customers, those organizations, then they start going, "Well, wait how do they work? What is their actual workflow? Can I make that better?"

If you make one improvement to the way those teams work, guess what, they'll listen to you. They'll beat a door down to you. You can't keep them away. Basically, if you make other people in the organization more successful, you never lack for work. You never lack for people wanting to be on your calendar, wanting to consult with you, wanting to talk with you.

If you are policemen, they will run from you. When you come down the hall, they'll dodge into another room and go away from you. You can't have the police state, the Stasi, the East German police kind of approach to UED. You've got to have that more than help solving problems. You're in that ride and then you get into conversations and you actually have skin in the game.

When you don't have skin in the game, you pitch ideas that don't really make sense. I've seen that over and over again. There was a team that got so excited about the pattern library. They were hiring outside the firm to build the pattern library. They got really excited about doing this pattern library.

They said, "We know we could have a chat in here so designers can chat with each other. We can have..." and they had so many features unlike us. First off, let's do an MVP. If we do a pattern library, how about the pattern detail page and maybe a rough taxonomy. That's it, if we do that, let's just ship that.

Applying the same LEAN start up principles...A cobbler doesn't have shoes for the kids, a design team doesn't apply the same design principles to themselves, is also often what happens. You have to take a real LEAN approach to those centralized teams and meet needs and get your work.

Jared: Where's Julie Martin? Elaine, we're going to go over to Julie next. Before we do that, one of the questions that I had following up with Eric is the politics. PayPal can be a political place, yes?

Bill: No!

Jared: Yes? No? No?

Bill: Shocking.

[laughter]

Jared: Is this a news flash? [laughs] Does it get political around design at PayPal? I bet it does. If so, how do you play into that? Has that changed as you progress through the different...

Bill: It has changed. We've had different actors - that's generally where it comes from. Actually, I have talk on like 18 or 19 anti-patterns that stifle LEAN teams. In those 18 or 19 anti-patterns, if we look up, Google that talk, you'll see me pointing out my real life experience of watching this happen. One of those is the genius designer because you get the person who generally they came from Apple or wanted to go to Apple with something.

[laughter]

Bill: They tried to apply, but work at Apple somewhere else which usually doesn't work. They're going to control the process. It always gets back to this if you could ever get a dose of reality of the customer into them. Problem is, this one person I remember that was in that role -- you know who I'm talking about, Karen -- would never listen to usability feedback. If they go to use usability and it's like, "This sucks." The users totally hated it.

He was convinced he had the right approach and just wouldn't listen. Fortunately, he left the company. If you can't get a dose of reality from the customer end, you're not going to be succeeding. When you get away from that, you fail. I don't know. I'm being simple minded here but it's...

Jared: It's something that you can do through hiring in terms of checking to see if that's going to work. I've been in the same situation, I'm currently working with a team where I'm going to start putting a jar on the table and if another member of the team says, "Well, when we were at Google, we used to do..." I'm going to make him put 20 shares of Google stock in the jar.

[laughter]

Bill: In your jar.

Jared: I'm just tired of that statement. I'm just wondering if the way to deal with that is to just whim them out in the hiring process.

Bill: I think so. When I look at patterns around design organizations, especially if you're in any kind of role of creating products, a big failure happens when you have design leadership who've never worked closely with engineering product. They maybe come from the studio background.

The studio background is fine. There are a lot of great things you bring from it. If you only have the studio background and you don't have the real world of talking to engineers, understanding engineers, what happens is I've seen those design leadership retreat back into their studio. They love to setup their own little studio and go hide in it. They don't connect and I've seen that happen over and over in almost every company I've been at.

You need to hire leadership that knows how to roll up their sleeves, and know how to compromise. Because design is not just art. It's the actual compromise with the real world, and very iterative, and failing fast, learning fast.

Jared: That makes sense. Where is...?

Bill: Julie.

Jared: Remember Julie is second, but where is Allan Ball? Which one? OK, over there. That's where we'll go after Julie, so Julie, your turn.

Bill: Hello Julie.

Julie: Hello. I think my company's process is very similar to how you described your organization when you got there at the design, with very long planning process where design doesn't start until afterwards.

Can you talk a little bit more about how you convinced the business to switch methodologies? Obviously, you can't just flip a switch, and have the business be on board with adopting a new methodology, and changing their process. to adapt more.

Bill: I was a little fortunate, although I've done it in other places, but I was a little fortunate at PayPal to have seen a leadership give me more carte blanche to change things. In other places, I didn't have that, and I also have friends who have been successful, or they didn't have that senior leadership.

What they've almost always done is carve out some sort of sandbox. Some place that they can make a significant impact, but it is not betting the whole company. I know like the folks at Hotwire had done this where they basically wanted to experiment with, "I can't show you where this hotel's at. Otherwise, it violates the whole idea of what we're doing. If I can put it on a map, in a general area, would that drive conversion?"

The sandbox didn't. When they first started, it was horrible. They were losing money, but they had got enough buy-in. They got a certain amount of dollars to invest in the lack of conversion, and then finally, they crossed it, and went way over.

Then it was like, "Oh, OK, great." Then, "How did you guys do that?" That methodology started picking up, and being used in other parts. You always have to have some sort of sandbox.

What you really want to do there too is I've always looked for a business development person, or a product person, somebody in the organization who's got an interesting problem, and hasn't been able to crack the nut, and you go partner with them, and figure out a way that you could try something.

They're generally always excited about doing that, because you're coming and giving them free resources for one. Helping them in any way. Then if you can provide some value, that gives you the right to provide more value, then more value, and then more value.

At Sabre, that's what I did, because even though I had the VP, and development who came in and said, "My charter was make us love usability," that was my charter.

[laughter]

Bill: I took and ran with it, but there, it was like grassroots getting to cross-pollinate teams. Trying to get this team that did crew planning, did the switch over, and evaluate the flat planning product. Teach them the discount, usability, methodologies, and then be more like a renegade sort of thing, which worked.

There, I found a few key product managers who had discovered where the levers in that direction had controlled. Teresa Neil, my co-author of my book, she and I just made them successful. Then before you knew it, we were involved in every conversation.

There was no single product being talked about that we weren't a part of. We were so busy we didn't know what to do with our self. That wasn't my charter. When I came to Yahoo, I used to say, "I heard you are the AJAX of AngelList?"

Well, they meant that to be internal, but then I flipped it to external too. I never stop with whatever the charter is. You get in, then you find out what the problem is to solve. There at Yahoo, we need to have an external evangelism too, bring great talent in, and then also speak to the whole design community like patterns, and stuff like that to help everybody get better. You just start solving a problem, and you find your way around kind of stuff.

Jared: Let's talk for a minute about this external evangelism thing, because I think this is something that in the field, we do scatter shot. How important is that to getting really great talent?

I think a lot of people think that the way you get great talent, and of course when Lou's here tomorrow, we'll talk about this in way more depth, is you write up that job ad, and it's funny, and cute, and you post it up on the wall, and that will get you the best talent.

Bill: Well, I knew when I came into PayPal in 2011, there was nobody in the front engineering community who wanted to go work at PayPal. Most had left. There are still some good people there, but a lot had left.

I knew my job really was to change the story. Humans love stories, and so as we started making those changes, as soon as we started having that Hermes project, I went out and started talking about it. People liked the message, and so before I knew it, I gave like 20 talks that year about the change we are underneath.

Jared: Silicon Valley in particular is so protective about their property, and their processes.

Bill: It helped that David was going out, the president, was making a lot of noise about that PayPal was under a big transformation. I got in trouble with our PR team a few times, but I'd already had...They grabbed me one day, "Hey, you need to tone that down a little bit." That was so trash talk, the previous PayPal.

[laughter]

Bill: It was all perfect.

Jared: You guys were trying to go IPO.

Bill: No, this is way, way before IPO. We were good friends, and I already make good relations with the team, so I could get a little buy. I said, "Look, I have to be honest. I have to go out, and tell how bad we sucked, because any engineer out there in the crowd knows it. It's no secret."

For them to hear an executive of PayPal saying, "We totally effin sucked, and now we're getting better. Here's how we're getting better, here's what we're doing," people love that comeback story. I tell the comeback story, and then before long, you had more people joining me who made the comeback story even more real, and more people joined.

It was a little self-fulfilling process, so I'd never shy away from taking advantage of telling a story and marketing, and those things like that, because people want to be part of something successful.

Jared: Allan. Where's Allan? Here we go.

Allan Ball: Hey, I just wanted to ask about how do you decide when to make broad sweeping changes as opposed to more incremental changes? Once you've made those broad sweeping changes, when do you do it again, or do you?

Bill: Well, it's interesting. I tend to almost always make incremental, and then at some point, you realize, there's a broad change to make. Then you make that change, because you learned enough to make a big change.

I probably would lead with the incremental, and then go to the big. Big is from your learnings. You got to reorganize the whole group. You've learned from talking to people, you've tried some ideas out.

The team that had been building the technology before I got there, the mistake they made was they had hired a lot of people, probably between 50 to 100 people, and they'd been working for about four years on this technology.

I came in, and ended up with seven people in six months replacing everything they had done. They were telling me the whole way through, "You can't do that. There's no way it's possible," and I said, "It's possible, because..."

They said, "You're going to fail." I said, "That's what you don't understand. I know I'm going to fail, but I'm going to fail in small increments. What you've done is you've failed in a large big gamble, and I won't do that."

I fully expect to fail every day, but I'm going to learn from it, and that's why we could do in six months with seven people what they couldn't do in three or four years with almost 100 people.

It's a powerful lesson, because I was listening. I knew what my developers needed, and what we've put out there, everybody wanted. I couldn't stop them from adopting it if I wanted to.

They had adoption team, and a second adoption team to help the first adoption team adopt the technology. It's just classic mistakes of big organizations. I can't make this shit, this stuff up. Sorry.

[laughter]

Bill: I don't want to put an explicit on your podcast.

Jared: Oh no.

[laughter]

Jared: We can have all that shit here. I'm cool with it.

Bill: OK.

[laughter]

Bill: Cool.

[laughter]

Jared: You don't have to go all Dave McClure on me, but we can...

[laughter]

Bill: I could do a [indecipherable 40:38] .

Karen: Looks like...

[crosstalk]

Jared: Where's Nick Cochrane?

Karen: Looks like we're out of time.

Jared: Oh. No, we have a little extra time.

Karen: All right.

Jared: Where's Nick? Nick has a great question after that. Where's Mickey Cancleve? Behind me. Perfect.

Nick Cochrane: Hi Bill.

Bill: Yeah. Hey, how are you doing?

Nick: My question is about the war room.

Karen: That's fine.

Nick: I work at ExxonMobil in IT, and we are trying to experiment with this high performing team concept as well. My question is about this war room. I know it was very successful, but I have a couple of questions about it.

One, did you face any resistance to what you were doing in that war room from other parts of the organization? Two, what happens after the war room? Do you roll into another war room, or is this more of kind of a unique one off project?

Bill: The war room was set aside by our president, so [laughs] it's like, "Good luck arguing with that one." In fact, when we would need a bigger war room, he basically told the facilities that if they didn't give us that most coveted conference room, the nicest one we had, he was going to give the board room up to us.

That's how much support we had, which was pretty cool, and he didn't have to do that, so he got what he needed. We then had a lot of copies. People were taking conference rooms offline, so much facilities was having a massive headache to create all these war rooms.

At some point, it has to get out of war rooms, and just become part of the normal process, and that's a harder thing to do. We actually were pretty successful at that. I would give us probably about a C at this point.

We had gone as high as A, and probably dropped down to C, and we're probably trending back up now, because as companies organize, reorganize as people leave, you've got to remake all those connections. You got to keep it going. I would say on the ground floor, the engineers and designers. product key people work really well together.

There are a few teams that it's probably a little rougher, but most teams do pretty well now. They imbibed this concept. We try to have a little bit more of a LEAN methodology we've put in place.

We had that for a little while, but when I reorganized my front end engineering teams, it became harder, because I know I decentralized that organization, and it became a little harder because they became a general engineering population. That became a little bit more challenging.

We've rebooted it a couple of times, and you'll have to do that in the organization. Just expect that whatever you put in place, it starts working now, it'll stop working in about six months, or a year, and you'll have to reiterate on it which was a big lesson for me.

Nick: The follow up on the war rooms. When you were in the war room, all these folks were in one room, and they've got the work on the wall. If they're leaving the war room, are they now in disparate places in the building, or are they all still stuck together, is the work still up on the wall?

Bill: What we try to do is we try to keep to putting teams in the same area. Where we don't do that, we're less successful. Frankly, it's just human nature. Once you get beyond what Christopher Alexander's called the nuisance distance, the communication drops off rapidly. Where people have to walk to a nuisance distance to talk to each other.

It could be on the next floor down. It could be on the other side of the building, which is why he advocated creating architecture to create communities. That's always a challenge though, because facilities is not as agile, and lean as the rest of the organization although David tried to make it that way. Tried to really do that.

We did break down all the cube walls, and make it open space and stuff, which helped.

Jared: Mickey was next, right?

Mickey: I'm wondering if you can talk a little bit about as you're pushing design thinking, or design methodology out, and especially up into your organization, how you continue to let your pool of talent feel really valued? There's this democratization of design happening in my organization right now, and I find that it makes the folks practicing it as sometimes uncomfortable as the folks learning it.

Nothing makes people more uncomfortable than the second an agency comes in from external, and was brought in by one of the executives you're trying to infuse with that design thinking. Talk a little bit how you keep both sides of the spectrum comfortable.

Bill: That's a tough one, because I've seen a couple of situation where we brought the outside firm in. Then the rhythm that the design engineering teams that were working together all of a sudden changed to where the design team was working with the outside organization and not working with engineering anymore. Then engineering was cut out of the loop.

I've seen that happen. What I've tried to do myself in those situations is intervene, and sit down and have the teams reset on what the problems we're trying to solve, how these different teams can pull together, and solve that problem.

If your leadership doesn't understand that fracturing is going to happen, then it's going to be a problem. On the ground, you've got to have to make a connection with that outside firm yourself, and try to solve the same problems together, which is tough, because they've been usually hired to just do some work, and throw it over the wall.

The other day, stuff has to get built. There are very few situations where you have an outside firm that's building your product. They are usually designing something, so at some point, your design team, engineering product team is going to have to take that and make something real from it.

That's where the real magic is going to happen, and so that's the best place to make sure the collaboration's happening in the most extreme case of it.

Jared: One of the questions that's come to mind is PayPal is one of the oldest companies on the web. Is there a mentality that's underwriting things of where people are just scared that PayPal could go away? That some new business model could come in, and just take over?

Kids on the west coast, they love that disruption word, though I don't think they know what it means, but they use it all the time. Is there this fear that PayPal will be a victim of disruption...?

Bill: Well, I...

Jared: ...and how did that drive, the design process?

Bill: Well, I think that they didn't have enough of that when I got there. In fact, if you look at Stride, it's been pretty successful from a processing perspective, they came about because the founders were trying to do a hack day, and they were trying to use PayPal, and the API sucked.

They said, "Well, we could do a better job than this." That's when they went off and created Stride.

Jared: Well, people moved from PayPal to Stripe...

[crosstalk]

Bill: I know.

Karen: I moved to Stripe.

Jared: Please. I know you got to rub it in.

[laughter]

Bill: You should have look at Braintree by the way. It's what I'm working on.

Jared: We can talk.

Bill: Yes. They didn't have enough of that healthy fear. I think that's definitely in the organization now. A lot of what we did to turn things around was because of the recognition of the Stripes and the Squares, and anybody else who was popping up. The Apple Pays, and whoever else.

The approach we've taken is that we're a fabric to the payment ecosystem so we can support pretty much everybody. As long as we have that mindset, we're actually more successful. Half of the Apple Pay implementations are actually running on our PayPal Braintree platform, so we actually enable that for a lot of merchants.

We've definitely got that healthy fear now, and the fear is really important, but you can't obsess over it. You've got to create real value for your customers. At the end of the day, that's what's going to win. It's not just focusing on them.

In our industry, payments is hard. It's actually really hard in industry, so that does help. You got compliance, and legally, you're in 200 countries. You know this, Karen, in MasterCard too. It's hard. It's a hard problem, so that's what lulled them to sleep I think, to think that they couldn't be disrupted, but anybody can.

Jared: What's the most exciting thing coming up that you can talk about?

Bill: I'm working on Nextgen commerce, which our next generation commerce products. I think what gets me excited is where commerce, and payments is going in the world, and how it can really enable people who don't have good livelihood now to in the future have easy access to money, to get loans, to get ability.

That what's happening in FinTech, we call it with the financial services...

Jared: Financial technology.

Bill: Yeah. With the unbanked, underbanked, and around the world I think is the most exciting thing that's happening, or going to happen. It's going to take a lot of work to get there, but just a little bit of money in the hands of people in countries that don't have it can make a huge difference.

Creating that equal access is, to me, probably the most exciting thing I see coming.

Jared: Well, that's great. I think that I'm excited about what you guys are doing, and where you are going, and the role that design is now playing in this, and so thank you very much.

Bill: Thank you. Appreciate it.

UX Advantage: Designing a Global UX

Gina Villavicencio & James Nixon - Gina and James discuss Marriott’s efforts in ensuring their brand isn’t lost across cultures, and how the organization is shifting to a more global perspective and locale-specific user research.

Jared Spool: A few years ago, I had the opportunity of seeing for the first time the receiving end of a localization effort. I was doing a project for SAP, where we were going off, and visiting SAP customers in the United States.

Those of you who ever had the joy of working with SAP are probably well aware that it's a company based in Germany. We were at one client, and the team that I was with, was the documentation team. We were trying to figure out how to better SAP documentation at companies that we're installing. 

We are sitting in a briefing with a client in Massachusetts and they were going through their switch over to SAP. One of the folks says, "Well, we've learned something in the switch over and what we've learned is, when you get to German, you've gone too far."

 [laughter]

Everybody in the room laughed, like you did. We all laughed but we didn't really know what that meant at the time, and I wrote it down. Later in the day we were sitting in a cube of a guy who's job it was to write reports.

He wrote reports for the company on outstanding credits, number of returns and all the financial things that the company needed. His job was going and taking variables that were stored in the database and putting them in report form. What we learned was that all the variables.

Because they were internal, were not subject to localization, so they were all written in German. Oftentimes, he didn't know what they were. But there was this extensive online help knowledge base, which the team I was with was responsible for. They were very proud of this knowledge base. 

They were very excited when he was trying to figure out something and he went into the knowledge base and he's clicking through in the knowledge base, and he's bringing up stuff, and he's clicking on more links, and bringing up stuff. They're glowing. They think this is the best thing ever. 

Then suddenly we get to a screen that's fully in German. Bam! The whole thing's in German. He says, "See, when you get to German, you've gone too far," hits the back button and we were gone.

That was the first time I'd ever been on the receiving end of what I've always known was corporate localization. The idea that, "Here we are, an American company. We do everything in English."

Then what we do, our notion of taking something global is pushing it over the wall and sending it to the localizing team. They're going to translate everything they can to what the local country is, and then we're done. That's it.

That's how we treated this notion of building global products for a really long time, but that no longer works. So what we're going to talk about now is what does work. To do that, I've got James Nixon and Gina Villavicencio from Marriott who've done an amazing job. So, James and Gina, come on up.

 [applause]

I was thrilled that the three of us were at a meeting back in January. You guys were talking about what you are doing with your global efforts at Marriott. I think that, from what I saw, you guys are really out in front of what a lot of companies are doing.
 
Gina Villavicencio: Thank you.
 
Jared: I'd really like to talk about for a moment some of, if you could share, an accomplishment or two that really highlight what being global means at Marriott.
 
James Nixon: First and foremost, thank you very much for allowing Gina and I to come and speak to you today about something we're so passionate about, which is digital globalization. We'll share with you a couple of product accomplishments we're pretty happy about that launched more recently.

One that comes to mind this year that we launched to multiple markets, was our ratings and reviews product.

The business imperative behind that is, first, we did some research and we learned that roughly two-thirds of millennials use ratings and reviews before they make a purchase decision when it comes to travel.
 
Jared: This is what we have up on the screen. This is the ability for people to share their experiences at the hotel.
 
James: That's right.
 
Jared: Your own private trip advisor.
 
James: That's right.
 
[crosstalk] 
James: You see it down in the bottom right corner on the front image. Essentially, two-thirds of millennials leverage ratings and reviews before making purchase decisions from a travel perspective. We knew that ratings and reviews were important for our search visibility within search engines around the world.

We also knew that Marriott's very well known here in the United States. In some international markets, they can't even pronounce Marriott. We knew that it was very important for those consumers as well.

Because when it comes down to making a decision, in terms of which hotel they want to stay at, they like to see reviews from people like them. For example, if I have children, if I'm a father, I want to see reviews from other fathers or parents to understand if this is a hotel that's friendly for my children.

We took that into account and we tried to figure out how to create a product that would essentially do no harm. We did some testing and understood that ratings and reviews could hurt conversion.

You're giving the opportunity for people to speak about your brand or your own experience. It could hurt unless we have a very strong brand image. We wanted to make sure we didn't erode any of the brand image that we had internationally.

There were a lot of things behind it, not essentially creating it and placing it on our site, but also the operating model behind it in terms of responding to reviews and making sure we curate them and all those things. I'm happy to say we landed on something that worked and is helping to increase our conversion.
 
Gina: This is an example of a feature that was designed by our domestic team, our headquarters' team in the US. When they put it together, they did conduct comparative analyses of websites that are key in international markets.

What you're looking at is an example of Qunar and Booking.com.

Qunar is one of the largest travel agencies in China, and it's owned by Baidu, which is the biggest search engine. It's an interesting player in that market. Booking.com, we all know who they are. The team created the designs partially with those websites with that in mind.

However, for that feature to be truly globally designed, we have to look at the customers that have less brand awareness of Marriott. Here, in the US, if I say that word Marriott, automatically, you may think of one, two, three words or adjectives that describe Marriott.

Internationally, users don't our brand. If this product was going to be branded Marriott reviews, does it make sense in other markets where they don't know us?

We had to do research in market. We had the funding to go to four different markets,Germany, Mexico, Spain, China. I have an example of what we tested it, because the source of the ratings and reviews, was key. 

In China for instance, we tested our reviews and then we tested the local version of TripAdvisor, which is called DaoDao. It was a qualitative AB test. What we found is that source matters, but not as much as the rating coming from people “like me.”

It was a key piece of information that those designs needed to convey. which was if I'm a reward member,in China, tell me “the ratings in from a silver member, like me.”

I was very pleased with the outcome of this project. We were able to domestically design it but internationally validate it, so that was a pretty cool story.
 
James: Do you want another example?
 
Jared: Sure, yeah.
 
James: Another example we're pretty proud of is the integration work that we done with WeChat. You guys are familiar with WeChat? That's good. I'm happy that many are familiar with WeChat. WeChat is definitely a force to reckon with. Please, once you leave here, take an action item to look it up.

WeChat essentially is WhatsApp meets PayPal meets everything else in China. It has roughly 549 million monthly active users. I believe that's 150 million less than Facebook Messenger. There's about a billion registered users.

Most of those users are Chinese. If you want to do business with them, that's probably a platform that you want to be on. It's only been out four years and already has a billion registered users.
 
Jared: I learned a bit about WeChat. It's this crazy platform. It's a text platform where people talk to each other.
 
James: That's right.
 
Jared: For instance, if you go to some local market, a farmer's market, you can pay the farmer with WeChat. You can send money over the text platform.
 
James: That's right. WeChat started out In China, Now, they've been expanding internationally and we've been interacting with them a lot, especially since Chinese consumers have been traveling so much, given the rise in the middle class.

However, WeChat did start out as a very basic platform in terms of accepting text, photos, videos, and voice. It was really cool. It expanded like they do in mini markets there.

Now you can pay everyday expenses like taxi cab rides, bills, invoices, so it makes sense at some point that you will also be able to pay for hotel stays and things like that or any products and services you guys have for your companies that are here.

We've been working a lot with WeChat in market in figuring out how to integrate it into our platform.
 
Gina: By the way, if you're downloading the English version of the app, it only has two features compared to the Chinese version. The Chinese version is an ecosystem of apps within an app. It's really, really interesting.

What I'm proud of is the way we worked with the regions in this particular project. This is the first time that the region is leading an initiative. In this case, WeChat platform and we're learning from them. It's awesome. We think that WeChat is what Facebook is going to be in a few years and this is an opportunity for us, the US, to learn about this huge platform. It's a very strategic way of looking at product development and design.
 
Jared: Snapchat already built in an e-commerce transfer mechanism. Facebook Messenger, there's been rumors of that for a while, this merger of moving money and moving text. WeChat is already there. What you're doing is you're experimenting in China with this platform as I understand it.

What you're learning there is going to be really useful. Your global efforts, it's the inverse of that SAP story I told where all the work was being done in Germany and then they localize it here. You guys are using that environment to learn stuff that's going to be the future.
 
James: I mean, when you think about it from a global perspective, it make sense. There's different markets around the world that are known for different things. In China, mobile, the mobile platform is equal or even more important than Web.

It behooves you if you really want to think forward and move forward from a design perspective or product perspective to make sure that you're learning from Chinese consumers. You best believe that's going to make its way into the United States and the rest of the world shortly after. 
Karen McGrane: Let me follow up on your point about the region's leading. Can you talk about where does globalization sit in your organization? How do you think about structuring an organization to be global?
 
James: Sure. Thank you very much. Global, first and foremost, starts with our CEO Arne Sorenson. A significant portion of our growth over the future is going to be coming from international markets. 

Arne strategically created an operating model that will support capturing that growth. Several years ago when he took over the helm, his first order business was to decentralize our company and to create regional structure where we have presidents in different regions.

He also created a mandate within our HQ organization to support their growth. Our brand team, for example, focuses on globalization of our brand. Our development team focuses on pipeline.

We, the digital globalization team, focus on creating relevant digital product and experience with our consumers around the world. More specifically about our team, I oversee the team and I report to the SVP of digital.

Our team sits at the core three priorities for our company which are global growth, digital, as well as millennials. We are both a horizontal and a vertical. Horizontal in a sense that we're the center of excellence that captures information from around the world.

Disseminates it to appropriate individuals within our organization. We're also a vertical in a sense that we build products and solutions that are global in nature across our mobile and Web experiences, but also localized products and services such as WeChat with some of the stuff that we're doing there.

Finally, in terms of thinking about change and some of the things that we've been going through within our organization, how globalization is changing. I think about a paradigm has three different dynamics which are people, process, and technology.

Specifically about technology, we will always think globally but act locally. We have done this to an intelligent decentralization approach where we centralize a lot of core functions that need scale. Our booking path for example is a great example of that. That doesn't mean localization aspects.

But we essentially can centralize it and push it out there at a large scale across organization. However, there's things that we also decentralize which are things such as WeChat example we were talking about or some of the local marketing efforts we need.

Quite frankly, they need people on the ground to manage that. Regarding people, the people side of the house is that as we continue to grow, the people in region will continue to grow as well. We need them to support our growth. We will grow at HQ as well, but mostly we're focused on scale there.

We need people in region. It's interesting, you would think that will be the same positions in each region. They differ a little bit based on the need. An example of that is in Asian markets. The language is a little more difficult.

As well as the Middle East with the double-byte character set, right to left. We need some additional resources from a language perspective to make sure that we are as relevant to consumers as possible in those particular regions.

Finally from a process perspective, process connects people to technologies. At the end of the day, it's about making sure that we have robust processes that allow us to continue to grow as we continue to grow in our markets.
 
Gina: The way that UX is structured...
 
[crosstalk] 
Gina: Given the product is structured as a vertical and a horizontal, the user experience team is also structured that way. We have UX leads that support each of the product lines. We also have a horizontal UX team which is our standards team.

It's lead by Mark Privet who's in the audience today sitting over there. Thank you, Mark, for your support, always. My team, the global UX team, which is a vertical and a horizontal.

Vertical because it supports the agile team, James' agile team, and also a horizontal because we work across the different product lines [inaudible 15:59] . My team is accountable for global UX. Each one of the UX leads are responsible for it as well.

My goal is that everyone in UX is accountable for global UX. Not just responsible but it's becomes part of their DNA. The moment that happens, my role may evolve into something else or may go away. That would be a huge win for Marriott Digital.
 
Jared: That would be a fundamental shift from this thing where it's like, "Oh yeah, we got to think about global" to taking it into account from the very beginning of any new idea or retrofit or redesign. 
Gina: Absolutely. It's a struggle. We try and we're still learning. Having people like Mark in the team that also has a global mindset is helpful.
 
Karen: Can you say more maybe follow-up on the localized prototypes about how you managed the competing poll of centralization and decentralization? How do you think about that? How do you get the local teams involved?
 
Jared: We have a picture.
 
Karen: We do. We have a picture.
 
Gina: We do. Great. The UX role right now, our whole practice is centralized. Right now we don't have plans to decentralize the UX function. What we're waiting for is to understand how some of the new roles in the regions are playing out. In China, we have a new product role.

We want to see whether that role is effective. We're allowing now authorship of content in the region. We want to see whether that's effective. If things like that continue to progress from a regional perspective, then we'll rethink whether UX should continue to be centralized. 

Also, we're looking very closely to user behavior trends that are happening in the market. That may also cause us to switch gears in terms of our structure. You look at a company, a product like WeChat. It's completely changing the way that people behave with their mobile devices. We have to keep an eye on that.
 
Jared: Say a little bit more about the localized content effort. How is that different than what you've had in the past?
 
Gina: Do you want to speak to that? The authorship of companies, what we've been doing on the hotel websites. We're allowing the regions to have more control, the type of content that we have in that product.
 
Jared: That's huge.
 
James: We're essentially figuring out an operating model that would empower the regions to have more control of their own destiny.

We've done a number of things in terms of localizing our home pages, in terms of our photography, the different marketing deals, as she mentioned on our hotel information pages we call hotel websites internally. 

We have started to open up a number of fields or regions that could focus on localizing hotel level content. We're going to continue to move in that direction because In order for the regions to grow, they need to have control of our platform in a smart way.
 
Jared: By moving or giving the regions some more control of their destiny, that feels like it's a big cultural shift in the US, possibly even more of a cultural shift for the US side than for the regional side. I assume that that shift has to happen slowly in the organization. 

Could you say some of the obstacles you've overcome already and how you managed to pull that off? It feels to me like once you start that ball rolling, it can start to pick up steam.
 
James: That is a great question. The globalization is definitely a marathon not a sprint. You need to have the support from the senior leadership at the top in order to make sure that it goes so effectively. The main obstacle that we've been having is fear.

Fear of what can happen, what if, loss of control in terms of our experiencing, and our brand as well which is a very valid fear in terms of how we're representing our brand and making sure that's accurate wherever it is.

What we've done quite frankly is we've picked off a number of areas that were important in terms of helping our regions continue to move forward. We've either conducted research like our example where we talked about ratings and reviews.

And our ability to be able to manage that on a global scale. Or we've created business cases. Or we tested things and we tried it out. Fortunately, I have a lot of flexibility working with the regional markets. 

We've been able to partner with them and do some intelligent experimentation and figure out and then prove to everyone like, "This can work. This does work. Look at the metrics.

Look at what's happened. Let's figure out how to formalize this and essentially scale it so that we can do this across our infrastructure. Let's continue to monitor and see what happens."
 
Karen: Can you talk about how you get executive support for these initiatives? I'm really curious to how do you get funding? Is it coming centrally? Is it supported by the regions? How do you get buy-in? 
James: Marriott has a very interesting funding model. I'm sure many other organizations do as well. Our funding comes from a number of areas, both headquarters and the regions.

One of my team's job is responsible for getting headquarters and the regions to align on our priorities for the following year. The better we do at getting them to align which is not necessarily an easy task, the more likely it is that we'll receive the funding for whatever initiatives we're trying to push.
 
Gina: James is completely understating how critical and key those conversations are. His team has done an amazing job bringing all the people together and getting them all to agree to spend a lot of money and that is beneficial for everyone.
 
James: The reason they're doing is because we're able to show impact that we've made.
 
Karen: Let me follow-up on that. How do they know if it's working or how are they measuring the success?
 
James: Globalization, a lot of it is with education. You need to help people understand how to think about different things. A lot of it is different in the way they look at it. They keep looking at things. The definition of insanity is doing the same thing and expecting a different result.

You have to get them to open up their frame of reference for how they look at the situation in order to understand if it's truly working or not. 

I think about three different buckets when we come to measuring impact.

The first bucket is the core metrics that we look at from a digital organization perspective and how all the efforts we do ratchet in those metrics. The second bucket has to do with adoption of digital globalization across digital team.

The third bucket we look at has to do with strategic bets that we place in market. Speaking to the first one which of the core metrics, Marriott, our digital strategy is very simple. It's to win the booking and win the stay.

Win the booking means Marriott.com, our mobile app, any of our experiences, we want consumers to book with us. Ratings and reviews is a great example of that, trying to win the booking. Show them reviews like me so I'll be more likely to book. That's great.

We do a ton of things around that on our experiences. The second piece, win the stay really is, when you're at our property, it's really tech and touch. Things like mobile check-in and mobile check-out.

Service request which we recently launched allows you to order things while you're hanging out by the pool or wherever you want to be enjoying our hotel. We're piloting things such as opening the door with your phone. The phone is a key.

In some of our hotels like the Edition in Miami which I went to, we have Netflix and other streaming things in there. Essentially, it's creating these digital touch points throughout our hotel experience to improve guest experience. All of these things are connected with the theme of personalization. 

That's really what makes it stick. We have a number of metrics that are associated with that such as in growing our room nights. That has to do with booking, the win the booking. Win the stay has a number of things like satisfaction scores and a number of other metrics associated with that. 

Internationally, we're much smaller than domestic in terms of our pipeline. Those metrics, although small, we have to show that there's large growth rates.

In terms of adoption, second one, as I said, we're a horizontal and we're a vertical. We're a horizontal in the sense that we're center of excellence that collects research and information and disseminates it to our organization. It's not enough to give people things. We also need them to use it.

The more that people adopt best practices that we have from a global perspective, the better our experiences will be for our customers. We have to ensure that our adoption rate continues to go up. Gina spends an ample portion of her time focused on this adoption which largely has to do with education.

I'd be honest. People are scared of what they don't know. Gina does a great job at helping to distill the information and taking it not from the theoretical but using the local prototype program to make it into an actual practical solution that people can understand.

She'll probably talk more about this. There's an 80/20 rule with globalization. 80 percent of the things are the same around the world. 20 percent of the things are localized. It's very basic.

The third thing in terms of strategic bets. Our group is essentially an incubator. We have a number of different strategic projects that we run around the world. One example is WeChat. WeChat which started off as a messaging platform probably didn't have as much implications for Marriott at that point.

Now that it has more of this payment functionality and all these other services, these start to tie into a lot of the things that we offer as a company and a lot of the things that we could connect to.

We make sure that we're strategically placed in a number of international markets on this type of platforms to ensure that once these platforms get attraction that they'll also be able to start contributing to our common metrics that I talked about in the first bucket. 
Jared: While we were talking earlier, Gina, you mentioned that you guys have come up with some really creative ways to keep global top of mind for people who are working on the vertical part of the designs. Can you talk about some of the initiatives you've been putting into play to...?
 
Gina: Sure. I'll be happy to. Global UX, what we're trying to accomplish there is to provide the most relevant and personalized user experience for our global users regardless of location, language, and digital touch point.

In order to accomplish that, we have several initiatives. The first one that I'd like to mention is on the product strategy perspective. That's one where our mandate is to be able to influence the product decisions not just for our international websites or apps.

But our domestic product as well as impact the platform, the Marriott platform that powers all of our digital experiences...
 
Jared: Can you give an example of a way that it impacts domestic platform?
 
Gina: Let me tell you a little bit about the prototypes and then I'll tell you how that impacts it. What we were able to do about a year-and-a-half ago was start these three localization prototypes. They're all responsive in key international markets.

We were working with regional design agencies. We were working with our regional teams. All these things are new, were new to us, to the UX team and it made a lot of people nervous. because what if the Chinese website needs to look completely different than dot com. Is that scalable? That would be really expensive. We had set out to do it anyway as the output The output of this exercise is what's different in content, features, functionality, Looking at the three across so that product by product line. Now we can say, "Hey, home page team, this is what we mean. This is what the research said. This is how you might approach it."

With regard to the example, there are specific enhancements within our search form for instance. In our Chinese and our Japanese search forms, if you go to our domains, if you compare it to dot com, you're going to see differences in the search form. That's because we were able to inform with research and design the local team.

That's a practical way of infusing global UX into the team.

The other one, and I'm really excited about this one is the work that we're doing with our standards team, so with Mark. Why is standard so important? 

Because as we continue to work with agencies domestically but we increasingly work with agencies in international markets, we need to ensure quality, consistency, and create exceptions when we have to.

The work that he and I are doing right now is super, super important. By definition, we've defined that a standard is not a standard if it has not been validated globally. Internationally, we have to go in front of users and understand whether our standards make sense.

I'll give you a specific example. Icons. Our standards take a core set of icons. Let's say favorite, a heart. Here, pretty much you can make the connection. The user could look at a heart and make the connection that it's my favorite thing or the most searched whatever it is.

Internationally, can we say that that's a standard internationally as well? Is a heart culturally relevant? Do we have to have the heart alone or the little icon alone or do we have to have a label to it to help with the interpretation? Those are the exercises that we're doing right now with the standards team.
 
Jared: When you do that with the standards...Is your goal to end up with one standard that works everywhere or a set of rules that says, when you're in China, you don't use the heart, you use something else?
 
Gina: The goal is to show the most relevant element to the audience. What I was going to say about standards is this is something new that we've been working on this year. Mark and I are going to be tied at the hip next year. I don't know if he will like that. It's pretty important work that we're doing in that regard. We're really excited about that.
 
Jared: We're going to take some questions here. Steve, over here, has a question. After that, Xihang? I didn't say that right. You're back there...get you mic. Steve, you go first here.
 
Steve: I have question for both of you, guys. Marriott is one of those companies that has the unique situation where your customers are, by definition, from a different place where the hotels are, which is different than a lot of us over here. How do you guys approach globalization?

Is it from where the customer sits normally or when they're traveling internationally? Is it from where the hotel is?
 
James: We look at globalization from a source market perspective, so where the customer is and they're leaving. When we have our different experiences, first and foremost, we have a road map that we've identified a number of languages.

A number of markets that we need to support which is tied to, if you look at it from a source market perspective which is tied to, looks at language, technology, looks at our pipeline in terms of our growth, we want to make sure we have over 90 percent of the languages covered from worldwide perspective.

We look at it there. We build a lot of our solutions from that perspective to support customers. Gina, you have anything to add?
 
Gina: I was going to say that if you think a market like Brazil for us for instance, we have four hotels there. Most of the business that we get from there is people traveling from Brazil to the US. 

And right now, we have a white label solution for that market. A key form of designing for them would be once we get them on platform is to allow them to pay , in Brazil for instance, you have a different way of payment. They all pay with in installments like they are used to. It's a very different way of transacting than people in the US.
 
[crosstalk] 
Gina: It's like a reverse layaway.
 
James: It's like layaway on steroids and dating myself again. I would got Kmart, dating myself again. It needs to pay for things. When you're done, like installment is interesting. In Brazil, it's layaway but you don't have to pay completely off before you get it.

You could start paying now, pick it up midway, keep paying afterwards. It's not just an economic thing. Layaway, at least in the US, I grew up in more of a middle class, lower middle class so we did a lot of layaway coupon, stuff like that. I got a sense, some of my friends didn't have to worry about that

People like to fly helicopters in Brazil because they don't feel like dealing with the traffic. They use installments as well. You have women buying Prada shoes on installment.
 
Gina: It's the way that they transact.
 
Jared: They're letting the payments up over time.
 
James: The banking system supports it. It's a nightmare from an accounting perspective. A lot of American companies going down there. Quite frankly, this is how the culture transacts.

We did an entire study on payments and how people pay, what they pay with, when they choose to pay, all these things differ by market. Some of the things we're focusing on now is the most important portion of commerce and e-commerce is the payment. We're focusing our efforts on dealing with that.
 
[crosstalk] 
Jared: How much of your work is educating the rest of Marriott on things like how crazy payments can get around the world?
 
James: Things like payment, that took us about a year to do, to convince the organization that this was important.

I would say over 50 percent of what I do is around education, is helping people understand and interpreting what we've seen and why this is important to them because people don't care unless it's important to them. Why is this important to you?

A significant portion of our, I don't want to say how much, but a significant portion of our room nights booked on dot com or US platform which is our global site essentially. It's our global site. A significant portion and growing every year are from customers outside of the United States.
 
Jared: This is becoming more and more relevant.
 
Gina: It is.
 
Jared: Where is Chris Hobart? There. We're going to Xi first and then back to Chris. Did I get your name right? I'm sorry.
 
Xi: It's OK. It's Xi, X-I. It's tricky, but it's all right. Thanks for trying.
 
Jared: You have my permission to pronounce me Shared...
 
[laughter] 
Xi: I think my English might be better than your Chinese.
 
[laughter] 
Xi: Thank you very much.
 
Jared: Xiexie.
 
Xi: You're welcome. I only posted one question on the portal. Is that OK that I ask two?
 
Jared: Yes. You earned that for me mispronouncing your name.
 
Xi: Thank you, parents. My first question is that when...It's a hypothetical question for you guys. If upper management don't want to go local, how do we accommodate our global users?

My second question is that, earlier in the conversation, James, you mentioned about making the team to adopt the idea of globalization. Can you elaborate on the steps that you took?
 
James: Sure. To clarify your first question, if senior management doesn't want support going local, does that mean creating teams in regions or what do you mean specifically?
 
Xi: We do have teams in different regions globally but it's mostly sales force. It's not the support that we have for our product. We do B2B network security. It's really intricate products. We don't have a lot of support team elsewhere.

Only the sales force and sales engineers are in global countries. Our main customer support are in the US.
 
James: You're talking about resourcing and marketing...
 
Jared: What obstacles did you have to overcome to make this happen at Marriott?
 
James: That's a great question. Keep in mind, again, globalization is a marathon, not a sprint, what I would try to do in your case is I would try to identify temporary resources or agencies or different resources I could work with in market.

It may not be full-time resources, but resources that I can get to try to run some type of pilot or experiment to prove from a business case perspective that by having this person on the ground, we are able to grow something by times X that we normally wouldn't be able to do.

Try to do that at least one or two markets to prove that this is needed on the ground in order for us to be very successful. That's how we approach things today. It's helped. You first have to get that initial support and some funding in order to be able to do that. How you do that, you may have to get creative.

The second question you asked was specifically around...I apologize...
 
[crosstalk] 
James: How the steps I've taken to get...
 
Xi: For adaptation?
 
James: That's a part of it, is to prove a case in a number of markets. Some of the things you can do, and some things may not even require resource being there. You may be able to do some experiments like for WeChat for example, start an account in China and manage that from the US.

Improve, show a business case on what you're able to do because I have this here. You could say, "Well, I've been doing this in my free time in the evening. It's best if we had someone to sit there and manage this on a regular basis."

Those are the type of things that we've done. Did I answer your question?
 
Xi: I think you did.
 
Jared: We're going to go to Chris here.
 
Chris: Hi. Thanks for the conversation. James, you mentioned that you desire for the regional sub sufficiency. One of the things you measure is adoption. When you're going through the globalization process, Gina, you've mentioned sometimes using data to say this data worked in this region, this did not.

Assuming there's experience or situations where you can't test everything, where you experience resistance to, "That template doesn't work in my region," those type of questions.

How do you handle that if you don't have the budget and funding to test everything or it doesn't align from a priority perspective to test that in this timeline?
 
James: Who would be pushing back, the region or headquarters?
 
Chris: Region.
 
James: The region. That's a great question. Your point is if you don't have money to test everything or prove everything, how do you get the region to buy-in and support what you're doing? That's a great question. Originally, we didn't have that. We had to build that.

Honestly, the way that we got around that is we have to prove ourselves. When I started, the people in China thought that Chinese people were from Mars and people from everywhere else come from Venus. We had to do everything completely different to support China.

There's a lot of things that are unique about that market which are fantastic. At the end of the day, people are people and there's a lot more things we do that are common.

The prototyping project that Gina talked about was our ability to do research at market and then to design and experience and test that experience with Chinese consumers. Through that, we were able to say, "Well, look, these are similar things that we do in the United States."

Through that educational exercise both at headquarters and the region, we were able to understand the differences and similarities. Through that exercise, we also built the trust in the region.

After we started to launch things, we showed the results. Now, the regions are like, "Hey, you know what you guys are doing. Go ahead and do it."
 
Chris: Thank you.
 
Jared: This has been fantastic. Thank you so much.
 
Gina: You're welcome. Thank you for having us.
 
James: Not a problem. Thank you for having us.
 
Gina: Thank you, everyone.
 
[applause] 

UX Advantage: Infusing MasterCard with UX

Karen Pascoe - Karen provides an example of what sweeping corporate change can look like when a company like MasterCard puts creativity, innovation, and its users at the top of its priority list.

Karen Pascoe: ...when you're dealing with mobile payments in particular, it's really around figuring out that you're you. That you can conduct the transaction. For instance, Apple Pay which is using our technology is really about securing that you're you. That you're financial institution knows who you are.

And you can tap and pay, and you can go, and you can be secure around all that. That's interesting. We're doing a lot of outreach into the developer communities. MasterCard is doing something that it's not really known for, not really famous for, which is opening up APIs, and software development kits to third party developers.

We're running a hackathon series. It's called the "Masters of Code." The grand finale will be in Silicon Valley, later this fall. That's been a 10 city tour internationally around. Really reaching out to the startup developer community, and introducing MasterCard's payment capabilities, security, flight capabilities, out to the world at large.

The third major area I'm working on, is really around mobile money. Personal payments is really what we call it, but we've got really broad applications for that. For instance in Nigeria, the national ID card is running on MasterCard technology.

Payments and identity, are really coming hand in hand. We're working a lot to really empower, and drive things around financial inclusion.
Jared Spool: You were brought in by the CEO, who before you got there was already sold on what you were going to do.
Karen Pascoe: A number of the executives that I spent time with...my first three months were spent on the corporate headquarters campus, which is in Purchase, New York. It's about an hour north, of New York City.

We work every day, or most days, out of what we call our New York City ted cab, which is near Union Square. I'm in 5th Avenue and 17th Street, right by Union Square, Farmers' Market, and Hip lunches, and things like that. I spent about three months getting to know the leadership, the various different divisions of MasterCard.

As I went around and met with operating committee members, or some of the executive staff, I was the most popular new hire and purchase. They said, "Oh my god. We really need you." Some of them said, "We had a hand in getting you." There were a lot of people claiming credit for my role, being created.
Jared: Why disillusion them of anything different?
 [laughter]
Jared: Because you have all this buy in from these major executives at the CEO level, what makes that different than other jobs you've had?
Karen Pascoe: Sponsorship is huge, especially when you're trying to drive change to set some contexts. MasterCard on one hand, is one of the world's top 20 most recognizable global brands. That's incredibly strong, to build on. I was the first strategic UX hire, at MasterCard.

All MasterCard really had, was a small usability testing team that sat within our technology in the organization. It's a sea of change to really bring user centricity, customer centricity, design thinking, right, service design thinking into an organization that has the customer at heart.

But often times, they think about the customer, they think about our merchants and they think about our issuers, but they don't necessarily think about the end consumer who's using the product.

Because MasterCard's a B2B2C company, so we work and we have the core products and all the payment rails and capabilities, but our issuing banks provide that credit card out to the consumer via the marketing and they handle the 1-800 numbers and things like that.

Working in conjunction with that, so being, while sponsored to drive this new way of thinking into an organization's huge, and gives it a lot of tailwinds.
Jared: Does that make your job easy?
Karen Pascoe: Oh, no. Driving change, Bill Scott, my dear friend, is laughing up here, because we've been through a whole change agent tree a time before.

Driving change in a large organization requires a lot of skills. You need to be evangelical. You need to be a subject matter expert. Sometimes you need to be a visionary leader. Sometimes you need to give somebody a kick in the pants. Sometimes you need to do some arm twisting.
 
But the most important thing that you need to do every day is roll up your sleeves and partner with people and align. Right? Get together and have a shared way of thinking. Figuring out how you're going to norm with these different skill sets and disciplines, working together.

How can you help them understand that your objectives are really aligned?
Jared: That idea of alignment is one that isn't a theme that we hear a lot in the UX world. We hear a lot of, we understand design, they don't understand design. We do the design thinking, they don't do the design thinking.

But alignment is very much more of a give and take type of thing. Does that make this harder or easier, particularly when you have the executive support? How often, it's like, "Oh, I'm not going to align with you. I'm going to call in the air cover from the CEO and we're going to do it my way."
Karen Pascoe: Some battles have been incredibly easy. At about the four, four and a half month mark, we were having some challenges in delivery, and we were really struggling. We were in St. Louis, where we've got a major operating center, you had two day offsite, I had dinner with my boss.

I said, "We need to go Agile. These are the five things that we need to do." That was the easiest win ever. Within two weeks, he said, "So it is written. So it's told. We're going agile, and we're three-quarters in right now, and we're really starting to see the change."

"We're past the first quarter where everybody curses behind closed doors, and it's getting far more effective." Easy, easy battle. Awesome! Harder battle is that our managers know how to manage in a waterfall context.
 
And they're used to making a demand around which quarter they're going to get what, with a fairly elaborate feature set. Getting customer centricity woven into our executives' fabric, decision making and management processes is going to be a journey for us. It's not they don't want to be sensitive to consumers.

They really do have the consumers' best interests at heart, but MasterCard really understands how to cut a major deal, and work it with very senior decision makers at an organization.

What my executives aren't used to doing is spending time in an usability lab, and saying, "Oh, oh. That's what's happening with the design. Now I understand why you need a little bit more time to work that through."

Those are the new kinds of muscles and DNA that we need to build into the organization, and we also need to build more technology awareness, and technology understanding, on the part of the executives, who've really had a different toolkit, historically.
Karen McGrane: Can you say a little bit more about how that transition, that agile has gone. You're what, two, three quarters...
 [crosstalk]
Karen Pascoe: Three quarters in.
Jared: How do you know when you're done? [laughs] Three quarters into the plan, or three quarters into reality?
Karen Pascoe: Three quarters into the transformation. I'll send some contacts from MasterCard, as an organization. MasterCard's big operations and technology processing were...We were on a big processing network, and we have multiple sites, but the center for it is in Saint Louis, Missouri.

Saint Louis, Missouri is not exactly a hotbed of presentation layer, customer-responsive development. We do have MasterCard Labs, it's been around for about six years, it's all over the globe, it's our R&D arm, where they're cowboys.

They do super-fast stuff, they hack some incredible innovations that my team really takes and puts that more into scale.

On the one hand, we've gotten the cowboys, who are great, and then we have the settlers who know a core system, but what we're building in New York City is this real muscle around being customer-responsive, being agile, minimum buyable product, ship, test and learn.

Getting those capabilities out there, and how to really apply those abilities to a business like MasterCard, where we've got two billion consumer accounts in 220 countries and we transact in 150 currencies over millions and millions of merchant locations.

How do we bring that innovation toolkit into a business where it's financial, it's regulated, it's got really big scale, and how do we manage it? It's getting our leadership comfortable with test and learn. Figuring out how we can do experimentation is really part of the learning.
 
Karen McGrane: You've had big roles in several other larger firms. What's different about this role? What's different about MasterCard?
Karen Pascoe: One of the biggest things is really leadership and culture. MasterCard has an incredibly successful business, but they were really slow in digital.

The leadership team said, "We're not really where we want to be in digital. We've got to figure this out." A lot of companies, when they go the route of trying to understand design, and putting it into their practices, they'll higher some young, mid-level designer with a lot of energy and hope for the best.

If they're lucky, companies will get these really good, strong people who have this incredible career opportunity to grow management skills as well as their design practice. God bless us, anybody who's gotten one of those roles. They're awesome for your career.

MasterCard knew that it needed to move big, so MasterCard took a step back, and they said, "OK. What do we need to do, in order to win in this space?" They thought really long and hard about where they were going to put innovation.

They didn't want to put it in California, because the corporate headquarters is in Purchase, New York. That didn't make any sense, and it was really about, "How can we attract the digital talent that we need.
 
So that we can have that formula that we've heard that works, and put the product people and the developers and the designers all in the same space, and let them collaborate?"

The thoughtfulness of the decision making was huge. When I was having my interviews, the first person who interviewed me told me about the spaces. They were looking at the space, and he was like, "You know, I didn't like that one. It didn't have enough natural light."

I was thinking out loud; I was like, "An executive is telling me how important natural light is in the space that they're going to select for the team." I was like, "Hmm, that's really thoughtful."
Karen McGrane: Steve Turbek talked a little bit about what it's like to work in an organization that has a really strong remit around the customer experience and how a user experience team fits into that whole puzzle that touches so many other silos in the organization.

Can you say a little bit about how that works at MasterCard, how that works for you?
Karen Pascoe: Yeah. My roles, my day job, as I see it, is really user experience and user experience design, which could be thought of as the design for the digital touch points. We think about it more holistically as designing the servicing experience right from end to end.

In February, I was tapped on the shoulder, by my boss's boss. He said, "I need to work on customer experience. I need to get that right." I've got a partner, Cindy Chastain, and two other people well-placed in the organization. He said, "You guys are it. You're going to help us figure out customer experience for MasterCard."

It's Customer for us, because it could be a merchant, it could be an issuer, it could be a consumer. It could be a digital giant, as we do work with them as well. It's also the interplay in-between customers and consumers and how do we facilitate that and make that happen.

We've been working on this for a while and have been reporting up to our leadership saying, "Here's all the strengths that we've got to build on, but here's the gaps." Customer journey mapping, customer journey management, that's really important, and we're not doing it. It's not because the teams don't want to do it.

It's that the teams don't know how to do it. Then customer experience measurement. If you don't measure your experience, you can't manage your experience. How do you know where to improve? As of today, we've gotten a bunch of headcount, re-prioritized.

And have been given a fairly sizable budget going into 2016 that we think we're pretty much going to secure to take the conversation forward. It's nice when a company puts its money where its mouth is.
Jared Spool: Because I've been having this conversation a lot lately, what is MasterCard's definition of the difference between user experience and customer experience?
Karen Pascoe: I'll use the more historical big company definition of customer experience as this yesterday, really talking about how user experience and customer experience are coming together. A lot of times, customer experience organizations have sat within operations functions within companies.

They're like the down 10 percent people. They're going to ship the locations of people to save on cost. They're going to optimize it, make sure that a cost center for a company is at the lowest unit cost possible. That's really the trade-off around customer experience.

If customer experience is run by people whose largely cost motivated, you've taken this channel of talking to and listening to your customer and you've said, "You're going to be reactive."

When you start looking at user experience skill sets and user experience DNA...the MBAs have been talking a good game around this for a while, but really, they're more strategists.

UX people really know how to make sure you've got a good experience from end to end and have that humility, and humanity, and empathy, and warmth, and activism as need be to make sure that that experience is better. When we net out at MasterCard of the definition of customer experience, it'd really be a fusing of the two.
Jared: You mentioned that you're in the process of building a team, so you're not like Stephen with 320 people yet?
Karen Pascoe: No.
Jared: What has that process been like for you, and how have you had to change the way MasterCard thinks about hiring to do what you think needs to be done?
Karen Pascoe: It's a really good thing I know a lot of people [laughs] , first and foremost. A really interesting time in user experience. I'm back in New York City for this role. I was in California at my time at PayPal. User experience has never been more on fire.
 
New York has never been more on fire, with all the start-up activity that's going on. You're not only competing for the best talent with digital agencies, you're competing against the hottest start-up, which a lot of the kids really want to start their career in a start-up these days for some interesting reasons.

You have to let them know that starting your career at MasterCard is going to be starting a world-class carer in user experience in a team that's not even a year and a half old. I've been doing a lot of work outreach with the community. I've been doing a lot of speaking. I'll continue doing that.

I've been working with the local groups UXPA and IxDA in New York to really foster relationships there and letting people know you're going to do good work here. It's going to be interesting. You're going to be well-sponsored.
 
We can provide a couple of things on the enterprise side that you really won't get in an agency. A spec that I tend to like to hire off of is I like to hire people out of digital agencies who want to do something else. They spent a fair amount of time there, they've got great mileage, their skills are really at a premium.

In addition to their skills, their tactical skills being at a premium, people on the agency side have excellent people skills, their consultative skills. Listen to what you need, not what you want. Being on the enterprise, you have an opportunity to have better work/life balance.
 
You have the opportunity to invest in people's careers over a long haul, and you have an opportunity to do things that...the agency business can be a fair-weather friend business at times.
Karen McGrane: Talk about how that translates into how you manage your team. Are you managed like you're an in-house agency? Do you have a charge-back system? How do you staff out people who work for you onto projects?
Karen Pascoe: My team is small. Right now, we're about 12 people. We'll be not quite 20 by the end of the year and growing a little bit more. I don't want to have 300 people. I really don't think I need it for what I need to do, and smaller is better.

Small with a great team, is always going to pay more dividends than a large and mediocre team. Sometimes you have a large, good team and you have mediocre processes that make it hard for great people to do great work. It's a good foundation. I, personally, am not a huge fan of the big charge-back model internally.

I work on a budget. We've been going through our budget cycle right now. I find out October exactly how much I get to grow next year. It's not quite clear, but I know I get to grow. That's all I've heard. What I do is I align my resources on the most critical strategic priorities for MasterCard and emerging payments.

That's what I do. If you're not the most important strategic priority, you can come to me, and we can talk about it, and I can pull in a vendor externally. I can look to do some staff augmentation, but my core, my base internally, works on our most important stuff.

The stuff that really needs to have staying power in the market. This is really about investing in the talent that's going to be able to produce outsize returns from an experience perspective because they know it. They've lived and breathed our business.
Jared: Who decides what the most important stuff is?
Karen Pascoe: It's pretty clear from my boss. He runs all the emerging payments. Our strategy's really clear. Our priorities are really clear. Aligning accordingly.
Karen McGrane: Can you talk a little bit about how you would use outside agencies, or how you would bring in freelancers, or other capability. Do you use them for strategic projects, and how do you assure they're aligned and don't walk out the door with all your valuable insights?
Karen Pascoe: Great question. How I've been using external agencies is going to continue to morph over time, and I'll probably settle it when I'm at somewhere between the two and three year mark, as the team gets closer to critical mass and scale.

When I started, I had no team. My first hire came onboard close to six months after I did, as I was getting acclimated. My onboarding process was long, and they wanted it to be long. They wanted me to get to understand the company, and the business, and the various operating divisions, and how they worked.

And get really networked, and get networked globally. At the beginning, I was using vendors because I didn't have a team, and the agency needed to be my team. It's starting to shift where, in some cases, I've got discrete projects, and then I've got agencies that are all ready under the procurement process.

Because we're a big company. We work with them. I've got some great partners on that front. That's helpful, but what I'm finding I want to move into, and some of our partners are more comfortable with this, and some of our partners are less comfortable with this is, we're going Agile.
 
That's it, Lean UX and Agile, a combination of that, so if I'm driving strategically what our experiences are and how they're coming together, I really want people who can work on a co-create basis with us, out of our studio.

They don't need to be there five days a week. They can have some time in their own nest, and that's OK, but given that, I really want vendors who are local, that work with us in New York City, which we've got a great capability there, but also who can work with us in a Lean and Agile fashion.

The agency business hasn't quite read Jeff Gothelf's book, or really taken it to heart, and they're still in the deliverables business. I'm in the outcomes business. I don't need big reports. I don't need funky little deliverables. I don't really want to pay for them.

I want outcomes that I'm delivering for my customers, and I really want partners who can work with me on that. We're at this inflection point of companies, like MasterCard, in it to win it, and really building these internal teams, and that's not going to stop.

The nature of how we work with our partners who have valued skills, and capacity, and things to bring to the table that are really important, how's that arrangement going to evolve over time?
Jared: The shift to Agile that you're doing, MasterCard's never done this before, so you get to dictate an Agile process that has design built in, versus the more common transition path, which is engineering, development, IT, decides that they're going to go Agile.

They use an out-of-the-book Agile process that never mentions the design once, and then you have to figure out how to work that. That's got to be a fascinating way. Have you been able to carve that. Your experience at PayPal, and other places, helped you learn those lessons?
Karen Pascoe: Yeah. My first transmission was at JP Morgan, and this was some time ago, so by the time I got to PayPal I already had a good toolkit for it. Then, at MasterCard, I was like, "We totally need to go Agile. This is a mess."
Jared: You need the new mess. The old mess isn't going to work. [laughs]
Karen Pascoe: Yeah. There are a series of Agile practices and cadences at MasterCard, that already exist, but I would say that they're low on the Agile maturity scale. People knew sprints, and they know stand-ups, and they know all the vernacular of it. What they're not necessarily doing is embodying the values of Agile.

Especially the values of Agile that are really critical to customer responsiveness. We're retrospecting on our major delivery right now. We're pulling everybody in, and so we're having some challenges on delivery. That's great that we're choosing to come together and retrospect on them.

When you're in a Waterfall shop, you have lessons learned after this really big, long, painful failure. Everybody spends a lot of time going like that, and finger-pointing, and that's not productive. They're highly politicized, and sanitized, and all that. It's not helping the teams on the ground learn in real-time how to get better.

With some of the recent challenges that we've had, we've instituted this cross-functional team retrospective, on a weekly basis, and it's getting good. It's getting good. It's getting productive. I've been setting the tone for it of, "What did we learn? How could we do better as a team?"

The culture of learning is really coming into play in the shared problem solving in between development and design. We have legacy systems and a legacy code base, so we've got challenges that we'll need to work through continually. But the spirit of partnership and the spirit of collaboration is getting far better.

Our developers are getting to the point where they're really understanding, "I can't do this for this state. I'm recognizing how important it is. Here's the things that we need to shift around in the priorities for us to get this done."

Those are all the micro-changes that are happening on the ground, which is really going to make us critical to our success overall.
Karen McGrane: Let's go to some questions from the audience. A reminder that we are taking session questions through Slack. If you're not on Slack and want to be, it's a great place. There's a lot of fun stuff going on there. If you need help with it, Lauren or Sean can help you, but that's where the questions get to come from.

The first one up is going to be Mark Privette.
Jared: He's over there, I think.
Karen McGrane: There he is. Steve Torbeck has one that we'll do after Mark, if you want to run a mic over to him.
Mark Privette: I'm curious what's the tipping point that made UX, or the whole CX picture, so important to the CEO of MasterCard?
Karen Pascoe: Our CEO has really been visionary in understanding that the digital transformation is a really critical inflection point for the payments business, and really getting that if you are going to have a customer experience in the age of Apple, and Google, and Facebook, and companies that obsess about the customer.

That was a really critical part of it. That's one part. The second part is with the rise of digital all around our business, there was certainly some learning that this is an opportunity where we really needed to invest to get better.
Karen McGrane: Steve Turbek is in the back there.
Steve Turbek: Thanks, Karen. This has been great. I have one question about, you're dealing with really complex environments with all financial services, but often there are usability problems with the product itself.

I'd love to hear how you guys are using user experience to make the basic function of a credit card experience with an end customer easier to understand?
Karen Pascoe: The physical card?
Steve: I mean like the ease, and rates, and stuff like that people have. In some cases, it's very complicated. I was very curious what you guys were thinking about.
Jared: I have a similar question which is when you're working on emerging technologies, is your user mostly the merchants who are using this, or financial institutions, or is it the end customer who owns the card?
Karen Pascoe: It's a really good question. I need to design for the ecosystem. With the way MasterCard plays in commerce, Fidelity is a customer, PayPal is a customer, Cap One is a customer, the first experience I might design, the first thing that comes to mind, the top set of use cases that I'm going to be looking at, are consumer use cases.

We might start with what a MasterPath branded experience might be, but quite frankly, the majority of MasterPath customers aren't coming directly, B to C, from MasterCard. They're coming from CitiBank or they're coming from our different issuers in all the different markets that we support.

When I go deeper, past the early conceptual phases and shaking it out through a couple of rounds of usability, then what we're starting to look at right now is really what is the provisioning process from a customer to a consumer? How do all those layers work?

We've been doing a lot of work around the checkout experience, which is really engaging with merchants, and what do we need to do for merchants?
 
In a way, I need to design for all of it. There's different parts of the puzzle that I'm working at various periods of time. It's really making it a win-win-win experience, that we're enabling our customers to deliver to consumers a secured commerce experience that's ubiquitous and works the same everywhere.

And it's highly secure, that they can use it at all the merchants that they want to to get things to enable their life.
Karen McGrane: I'd like to follow-up on that tangentially. One thing that I've heard people say about applying for UX jobs in the financial services industry is that it's all super-wonky, boring stuff.
Karen Pascoe: [laughs]
Karen McGrane: How do you recruit for the right people, particularly if you're recruiting out of a market that I know, which is the New York agency market. How do you find people who not want to do that work, but are good at doing that work?
Karen Pascoe: Who's your selfies? A show of hands if your selfies. Nobody likes selfies? We're doing some really interesting things around biometrics. We've taken a stage in a company that takes a biometric based on the unique signature of your pulse.

We're obviously doing work with fingerprints, which are known biometrics, and voice, and other things, but the latest innovation that we're in pilot right now is what we call selfie pay, or the hashtag on social feeds or around it, so really interesting ways of identifying the consumer.
 
The consumer wants to authenticate a transaction that they've done, then they take a selfie, they need to blink, and it goes and matches and verifies to their picture on file.

This is future stuff that's happening, that we're doing, so that when people start to think about payments and commerce experience, it gets a lot different than what might be a traditional financial institutional/mobile banking context.

And they see the importance of experience into our business, overall, and really tend to see the opportunity.
Karen McGrane: Let's go to Katy, right here.
Jared: Where's Katy?
Karen McGrane: She's right there.
Jared: While we're going to Katy, you mentioned earlier about the executive who was super-concerned about natural light. That's strikingly different than many executives I've met, who really are super-concerned about keeping people in the dark.
Karen Pascoe: [laughs] Strikingly different.
Jared: Yes.
Karen Pascoe: Strikingly different.
Jared: Katy?
Katy: Before we had online services, a credit card was really marketing campaign, piece of plastic, statement in the mail. I'm curious how you've worked to integrate UX with the traditional customer experience functions that exist within MasterCard and how your message is resonated there?
Karen Pascoe: A lot of times with us, we don't issue the credit card. MasterCard doesn't issue credit cards to any consumers. That all happens through your issuing bank. We're really taking a look at the notion of making you digital by default.

We're doing a lot of work, and we will be doing a lot of work, over it's probably going to be about the next year-and-a-half, maybe even two years, is really looking at our core consumer base that gets issued this plastic card.

A plastic card is really representing your account credentials. It's what you have, the ability to pay, based on what your credit worthiness is to the issuer who's got your account with that, in conjunction with MasterCard.
 
And the processing network, and all that, and Apple Pays, MX, when you tap and pay, or Android Pay, or Samsung Pay, or any of the pays that are coming out, that are riding on our rails.

We think that there's a really great opportunity to understand this end-to-end experience in the context of how the physical world and the digital world from payments are really starting to come together.

There's an opportunity to educate consumers who want to understand online commerce in a more seamless, streamlined and safe way that we're really shooting towards with a lot of the technologies that we're working on right now.

We're really looking at, philosophically, for a North Star is, "How can we make the digital world as safe and seamless as the physical world?" From being provisioned a card, to being provisioned your digital capabilities, to using your digital capabilities, to getting reported on.
 
And to having something that consumers think is critically important, which is really financial control. How do you have control over what you're spending, and making sure that you can monitor things effectively? That's a longer conversation that we're looking at that's really interesting.

Where can you rethink a physical touch-point, and make it a digital touch-point, but not from a cost-savings perspective, from a consumer-benefit perspective.
Karen McGrane: Let's go to Justin Powell next. Where's Justin? Oh, there he is.
Justin Powell: This question is about the most important step. Jerry hinted at it. I was curious if your team helped support, because sometimes these priority matrixes don't have an experience vision casting, or something like that.

I didn't know if you guys supported upward, or if you told, "This is the most important step."
Karen Pascoe: Oh, yeah! We're totally supporting upward. Let me explain that a couple of ways. We're still a fairly new team, and I'm still lighter than I'd like to be on user research. You have generative user research, genitive research and evaluative research.
 
We're doing a lot of evaluative, we're doing a lot of usability studies, and we've got the wheel going on that. We need to do more generative work overall. The more we build up insights, the more we're driving the conversation around the what. That's a big part of it.

Then the second thing we're doing as...I'm starting to hit critical mass with the team. With our agile transformation, we've really got two sets of user stories that are running. The first is combine, where, how we're defining it at MasterCard, it's the open-ended stories. It doesn't need to run within a two-week sprint.

The teams can take them and elaborate them in an open-ended fashion, as long as they need to, and when the teams are ready to flip it over into a sprint, then it's the hamster wheel that you're running and racing in your sprint to get it done in your two-week cycle.

Having the blend of the combine allows us the opportunity to elaborate the stories, and that can drive into a level of prioritization, and we can say, "They really like how this worked out."

We've got the opportunity to really get ahead of it, and really drive the direction of the product, or take a time out, or pause, or keep it in combine until we feel like we've got it in right, in terms of what we need to do.
Karen McGrane: Let's take one last question from Collin.
Jared: While we're waiting for Collin, I was curious. How hard was it to get executives to sit in on usability tests the first time?
Karen Pascoe: We haven't yet.
Jared: Oh, OK.
Karen Pascoe: Yeah. That's a work in progress. We're doing some guerrilla. We're taking into it. We have a usability lab that's been blessed in our space, we've taken an additional floor in New York City, and that's going to come online about midyear in 2016, maybe the second quarter.

We haven't subjected them to it yet, it's really been the product executives who've come in, but I can't wait. We're putting the usability lab on our most prominent floor in our New York facility, where we take all of our customers around.

It's going to be super easy for me to run studies when I know that they're going to be in town. It's going to be a real game changer for us. That, and having it in our facility when it's set up.

Right now we're doing lab testing, we're doing some guerrilla testing. It's really hard to get our execs there on an interim basis, but when we have the facility in New York, our developers are going to work out of the lab.

Whatever Scrum team's working on it. We'll be able to pipe it into Saint Louis, Purchase, London, any of our other major operating locations, to have multiple viewing sites through our telecom.
Collin: It sounds like MasterCard has a lot of great new UX initiatives that are new to the company on the go. I was wondering how the company measures its return on investment for building brand new user experience teams, making user experience design a priority in ways that it never had before.
Karen Pascoe: I don't think that MasterCard sees it in the context of user experience in isolation. There's probably a couple of terms that could be used interchangeably that we're not really cognizant that we're using interchangeably.

Customer centricity, customer responsiveness, innovation, user experience, digital. All of those terms really mean, "I'm going to clearly understand a customer paying point. I'm going to work iteratively to take that customer paying point, and make it better."

That is supporting our digital efforts overall. The investment for MasterCard, located in Purchase, New York, to take three, now four floors in New York City, to staff up and collocate presentation layer development, some full-stack engineers, user experience, user research.

Our MasterCard labs has capability in there, as well, as our product teams, and really put all those functions under one roof. Four floors in Manhattan, I'd say that's a pretty sizeable investment. It's really reinforcing that digital is an area for us that we really need to win for our business over the long haul.
Karen McGrane: With that, thank you. This has been amazing. I really appreciate your coming out.

UX Advantage: “Organizational Becoming” Made Practical

Marc Rettig - Creating a culture of design is a special case of “organizational becoming,” touching teams, processes, and the delicate and difficult areas of culture and identity. Few of us feel equipped for such work. But “social technologies” are appearing to help us manage cultural emergence. Weaving theories of change with the nitty-gritty of daily life, Marc uses stories from organizations large and small to describe the ways and wisdoms of becoming an organization that embraces design.

Marc Rettig: Thank you. I love being here. I love that these conference exist. Thanks to the team for that. Just to get it out of the way, this is the three hats that I wear these days, my affiliations and my contact information. I'm really happy to talk with you about any of these things, about the content of my talk. I promise to try to be responsive if you send me email or reach out. We should just get started.

I've written this talk in response to the questions of the conference, which are right there on the front of the website. What does it take to create a culture of design? How does putting user experience first changed the way organizations work? Those are great questions that I'm glad we're finally talking about them.

Really, this talk is me speaking to you from the bottom of a well, or the corner of a meadow or something. Unlike a lot of you, where your work is really fast phased, I stepped back for the summer, my colleague and I, to reconnect to the work that we really want to be doing, that we chose to. I'm in a reflective state, so that's where this is coming from.

I will have some specific stories, but lot of this is high-level view or high or deep-level view. So it's my job to put the ideas out there, and your job to see if they make sense and put them in conversation with what you're up to.

Let's start with a fable. This is the fable of the shoemaker and the strategist. In a village not far from here, there was a shoemaker. He was very good at his craft and he was very happy. In the evenings, he would stroll through the town, or maybe, people would come visit him in his shop. He would talk with everybody.

So he knew their habits and their shoes and their cycles and their rhythms. He knew their needs. He knew the dreams they had for their shoes. He would know, for example, that school was about to begin, and that the children would have outgrown their old shoes.

As fall came, he would start to work on heavy winter boots of many styles and sizes. In spring, he'd speak to the brides and grooms about their wedding shoes. In short, the shoemaker was part of village life and in tune with village life. He's in a kind of dance with village life, and so he made his living.

Now, the shop was owned by the strategist. Until recently, the strategist had been very happy to be in partnership with the shoemaker. He counted himself lucky. All he had to do was order supplies, keep accounts, see to it that the roof didn't leak and the bills were paid, and sometimes, yell at the apprentices.

He paid the shoemaker, kept a little for himself, so he made a living. But the strategist was having an idea. "The world is changing," he said. "A department store has opened, and they sell shoes much cheaper than we do. Their shoes are much the same and they are thinner leather. They refuse to fix the stitching when it flays. But still, we must learn from this. We must react."

So the strategist went to the shoemaker and told him his idea. "We'll hire more apprentices, we'll put them to work making more shoes, all the same. I'll order thinner leather, I'll order in bulk. And we'll sell our shoes in other villages too, our neighbors all around the region. So there'll be lower cost, more shoes and great profit. The department store has shown us the way."

The shoemaker was silent. At first, he was numb, then he was sad. Then he was angry. Would the strategist have him abandon the people of the village? And leave them to get on with the department store shoes that wear out so quickly?

The shoes would be cheaper, but they wouldn't last as long. So people would wind up spending more of their precious money on shoes. How could the strategist consider such a thing? So they shouted and argued and went to bed without resolving their difference. Continued the next day and through the week, alternating between loud, strong words and loud, strong silences and loud, strong glares over the top of their glasses.

We're going to leave them there, I'm sorry. We're going to leave them in the middle of the story, and we'll come back to the shoemaker and the strategist later to see what happened to them. It's just a fable, and I know it's stereotyping. But I'm trying to offer two stereotypes.

The shoemaker sees the world as a village, and sees himself as part of its living fabric. He operates by spending time with the people that he creates for. He knows their cycles and rhythms, he sees himself as part of a larger dance of life. He's doing his part to make it better, and receives a living and a lot of enjoyment in return.

But the strategist sees the world as a continuous interplay of players in a game or people in a battle. He sees himself as a lone actor who relies on good ideas and good plans to bring prosperity to himself and to those around him. It's not all about him, prosperity to himself and those around him. He seeks to control things to see that the ideas and plans are well executed.

Just to, like, put it in words instead of pictures, the shoemaker thrives on connection. He pays attention to the village and acts in direct concert with what he learns in a kind of dance. His customers are real people that he's actually met. The strategist thrives on control. He loves clever ideas, and he employs planning and execution in a kind of march, and his customers are fungible abstractions. Now, I don't think I succeeded. I tried to write these without judgment. These are...

[laughter]

Oh, yeah, clearly I did not succeed. I'm talking to a roomful of designers, too. These are both legitimate ways to participate in the world. Yeah, and it's all over the place. But let's leave the fable. We'll come back to the fable. But let's leave it for now.

We need some definitions. If we're talking about design culture, maybe we should know what we mean by design and know what we mean by culture. By the way, so I'm going to offer process definitions. I'm not trying to be definitive or all-encompassing. I want hard working definitions, definitions that we can actually use to guide our engagement with these things.

Let's start with design. Meet Hugh Dubberly. Hugh is a well-known fixture, consultant, educator, kind of an all around mover of things forward in the world of design, especially in interaction design, design education and strategic design.

He has made a number of very thoughtful and beautiful conceptual models which you can find on his site, his firm's site. He spends a lot of time digging into whatever it is he's paying attention to and then a lot of time working out, what are the essential components in relationships. Then he spends a lot of time iterating and making a presentation of it that helps us all understand.

A few years ago, Hugh noticed that a lot of people in a lot of different fields seemed to be doing some kind of iterative process. He dug into them and he made little working models of these things. For him, the design process is a special case of a much bigger idea, which he calls the creative process.

Here's Hugh's poster, it's a great big poster, you can download it for free. Somehow people get printed copies of this. But that's some pretty fine print. It's really great, it's really good reading. I recommend it. We assign it to students, we do exercises around it.

But I want to zoom in on his language for the three main loops, observe, reflect, make. Which you'll notice are themselves loops and which connect to their neighbors by even more loops. But I want to, if we're going after a working definition of design process, a process definition of design, as an example, expression of the creative process. Let's zoom in. We're going to read these together. You'll just have to follow the imaginary bouncing ball.

Observe. Observe through conversations with context and people. Observe with attention, acting with respect and mindfulness, contributing passion and energy. Observe with openness. Listening and learning from other people and other cultures. Observation begins as a conversation with others.

Isn't it interesting? Observe through conversations, I think, already we're in some profound territory. First you're on the outside looking in, slowly you immerse yourself. Then you can step back and reflect. Where are we? Who is here? What are they doing? What are we doing? What's important here? Why?

We're looking for a useful definition of design. Yeah? Here's someone who's thoroughly studied, he's actively practiced in industry for decades, he teaches at the grad level, so he influences the future of the practice. He's learned directly from past masters.

This isn't Hugh opining or preaching, this is Hugh documenting the foundations, he's passing on documentation to us. He's telling us that paying deep attention to the world and getting into real conversation with it with openness is one-third, it's the first third of the essence of design. As in, if you aren't doing this, it's hard to say you're doing design. Maybe you aren't doing it well.

We have a question in front of us at this conference. If this is part of design's essence, what needs to be true about an organizational culture in which good design can grow? What needs to be true about our cultures for them to embrace the observe aspect of design?

OK, so that's observe. Here's reflect. Having observed, reflect through conversations. Reflect through conversations with experience. Reflect through conversations with the past and with values, with the things you care about. Reflect through conversations with your experience with the past and with the things you care about. Reflect to understand what people want and how culture is evolving. Reflect, to integrate by seeing patterns, and by building consensus. Integrate, by building consensus.

Reflection begins as a conversation with one's self. It starts to close in. It considers experience and values. It frames a situation or selects a metaphor to explain it, which must then be shared with other people.

Now, we have a question again. Reflection is part of the essence of design. Reflection is essential to design. If this is part of design's essence, what needs to be true about our organizational culture in which a good design practice can grow? What needs to be true about our culture's for them to embrace the reflect part of design?

Make. Make, through conversations. Conversations can make up a lot. Make, through conversations with tools and materials. It's a generous definition of conversation. It's not two-people saying words at each other. It is bringing what I bring into an encounter with the other.

I'm offering something, and it's offering something. We're having an exchange. I would prefer dialogues to conversation maybe. Make, through conversations with tools and materials. Make, to search working quickly and iterating, taking advantage of accidents. Make, to envision, imagining the future and making it tangible, explaining what the future might mean. Making also begins as a conversation with one's self. As it continues, it increasingly involves others.

I'll say it again. If this is part of design's essence, what has to be true about our organizational cultures to be a culture in which good design can grow? What has to be true about our culture is to embrace the make aspect of design.

Here's the loop. If we are saying we would like to foster design culture, we're asking, what does it mean for these things to characterize our culture? Notice that every one of those definitions involved conversation. Design is conversational. Every step's conversational, involves some kind of collaboration and requires some kind of openness.

We have a start on our question, what does it take to create a culture design? It takes open attention, and it takes real conversation. We've watered down this conversational definition. It takes open dialogue. It requires open attention and dialogue with context, with people, with experience, with values, tools, and materials.

There's basically a whole design curriculum in this one slide. We could work students pretty hard from this one slide. I can still carry this phrase. This phrase just blows to my brains sometimes over coffee. Quality of attention determines the quality of result. Take a breath. That was design. Now, it's culture, because that was heavy.

It's kind of a layered definition, a working definition of culture. Here's form. All of us are organizations or ourselves produce some kind of result in the world. It might be digital. It might be an experience. It may be physical, but it's the result of our efforts. Where does that come from? It comes from somewhere. It comes from structure and process.

The way we organize and the way we work together, the way we work, the way we work together, the way we organize. All of that gets busy and out pops stuff. What we don't often talk about is, where does that come from? There's a conversation in the room. We say, "Hey, is this thing ready to ship?" People are like, "Yes, totally ready to ship."

I agree. It's ready to ship. Who are you? [laughs] What are you drawing from, when you agree together that it's ready to ship? When you say, "Is it design be at the sea level? Is that where its home within marketing or engineering?" It split up. It is distributed, and then you think about that. Then, you come to some agreement about your organization.

Who are you? What are you drawing from, when you make that agreement? It comes from somewhere. That's a place we don't talk about. It's a blind spot. That place is culture, the essential driving story, the myth of our existence. It's got part, the identity, who we believe we are, what we care about, how we relate to others.

Sometimes, it gives people trouble, so I gave an example. To take us on our suppliers, do we stand in power over our suppliers? Are we in partnership with our suppliers? Are we dependent on our suppliers? Are we bigger, or equal, or lesser than them? That's a cultural relationship question, an identity question. Explanations, how you believe the world works.

By the way, there can be some monsters in here. There can be things like, "I'm not good enough. We're not as good as they are." Just a crippling sense of isolation are not good enough. You can feel that way. "That guy is such a great leader. I will never be as good as that." Down in our identity layer, there are monsters as well, which is one reason people are afraid to look down there.

That is a useful listing of components of culture. Let's ask about the shoemaker. He may not be able to articulate this. [inaudible 00:16:54] we can? It's down in the place, where there are no words. It's just part of the shoemaker that A, "The world is a fabric of craft and care. My role is to cover, protect, and decorate the feet of the village.

When I'm in my place, the farmers can do their chores. The children can learn. The blacksmith can do his work." That's a confident worldview he's got. He sees himself as woven into a fabric. He wouldn't use those words, if you ask him about it.

The strategist. The world's made of exchanges of values. Some people produce much. Others produce little. I participate in a great eternal game of accumulation of wealth by being smart about how I play that game. Again, he may not say it that way. Maybe if he was drunk.

[laughter]

It's there. It's part of how he decides whether he is doing the right thing.

OK, having gathered some working definitions. We did a series of interviews early this summer, I guess, late spring, getting a picture of the efforts of managers in large tech companies who are trying to build UX capability. We had a peer, we had a kind of a, friends in Japan who were doing the same kind of thing.

It was like this benchmarking light, just kind of a glimpse into the world of trying to cultivate UX in these huge, bureaucratic, sometimes kind of a military engineering, historically engineering cultures. Yeah? Here's something I noticed that, we'd hear people say that they want a design culture, but what they really were working on was design capacity. I'm finding that a useful distinction.

Let's talk about that. Because I think than not, a similar difficulty might be in there. To have design capacity is to have the knowledge, the skills...This is an exhaustive list, you get what I mean, to have the knowledge, skills, tools, people, places, processes, structures. You want to be able to apply the design process in your part of the world, so you're building a practice.

The reason is you want to produce better results and service to your cultural story. Better things. Better forms. Let's put design in the organization structure and process layer. Let's put it in there, and we'll get better, shinier things, and people will like us more.

I'll also say, most often, even though they're trying to put design process into the layer there, they don't have as deep or hardworking a definition of design as the one we just discussed, never mind that that sits at the roots of design history [laughs] and the best practice for decades.

Some reason, design is making its way into industry with a case of anemia, but I'll stop whining. Whatever your definition, the point I want to make is putting design capacity in place is not the same as putting design culture in place.

This is a way to caricature that we're moving from not only have linear decision making, kind of a manufacturing model, to also have iterative refinement in our process toolkit. The same intention, the same values, the same understanding of the world, but a new way of operating layered on top of it.

I want to use this little diagram to talk about the consequences of that, the observe-reflect-make loop. This represents just one of those turns. You can put these end-to-end and off you go. This is the grand version. Of course, there's lots of little loops, or hopefully lots of little ones inside this. Let's just use this as a tool.

Here we go. Let's look at a picture of design capacity without design culture, design process installed on the old story. We send the researchers out. Why would we go out ourselves? It's ridiculous. Send the researchers out, and they bring us back a set of themes, and insights, and opportunities, because we have a process for that.

They saw orange. Orange stuff is coming into our company because of our process. Then comes the management meeting. What should we do? How do we make sense of this? That's when it encounters the culture. Orange came in the house. What do we do? Geez, orange. The thing is, we don't do orange. We do blue.

If you could take that, all that you learned about orange, and use it to make better blue, that would be great, and we have a process for that. We have a process for that. It's make part of design. The result is that people are frustrated.

If you are the person in charge of establishing design capacity and you're measured by things like ROI or the impact of, "We did design, and where's the return?" it's maddening. Maybe the world has a different thing, but the organization has been unchanged by the experience.

Design capacity without open dialogue with the organization's underlying cultural story, identity, values, relationships, and explanations will produce leaden results. Any stuck story is dangerous in this changing world, and any stuck repetitive cultural story...I know that I'm stereotyping here, and I know I'm preaching to the choir.

The story of more revenue is why we exist, or market share, or beat that competition. Those stories are too small and mean to serve as the soil of a thriving culture of design.

Even innovation, we make new stuff that's cool, and people really like it, and they buy a lot of it, even innovation, I would say, is too small and still a little mean to serve as the soil of a thriving culture of design. I say that why? Because they're closed stories. They're inward-turned stories. They're about you, so you repeat your own pattern.

The world will change, and you risk a crisis of irrelevance if you don't let it into your story, into your culture. Opening is such a big, important deal in the definition of design and in this example we just saw. I want to dive into it a little bit. Opening. I actually find these images really helpful. These are also things that are helpful reminders. We use them with students.

They may be abstract, that's pretty abstract, but it's fundamental. It comes from Otto Scharmer from MIT and something called the Presencing Institute. This eye is your center of attention, or your team's center of attention, or your organization's center of attention. It works at all scales.

The circle is a kind of screen or bubble. Some people, you can see it as a transparent bubble that holds all your judgments and beliefs, so when you look out at the world, what comes in is filtered by your beliefs. Only some things make it through. Other things don't. Famously, we dismiss data that disagrees with our assumptions.

Another way to see this, some people like this metaphor better, that it's a screen, and we project our beliefs, and stories, and explanations, and values on it. We walk around seeing the world overlaid with our projects. We're sort of seeing both at once.

This is the enemy of observation, judgment. That bubble is the enemy of open observation, the enemy of attention. How do you conquer that enemy? You conquer it by learning to move your center of attention. You move your center of attention so you peak out past your judgments and past your personal lenses.

Learn to see with innocence and wonder again. It's actually not that hard to learn. You can practice this. It's not that hard to learn. Probably some of you teach people this, too. Even teams in organizations can learn it, although the conversations get harder as the scale goes up.

There's an enemy to this step of wonder, of moving your center of attention, and that enemy is cynicism. Cynicism is a product of repeated disappointment. "We won't be able to do anything about this." "No one will believe me when I tell them this story." "We've tried to change that situation." "I see it, oh I see it, but why engage? I'm not persuasive enough. The bad things about the world will soon squash this beautiful scene I saw." [laughs]

The way to conquer cynicism is another move of your center of attention. Move your center of attention outside yourself. Now, you can see that it's not about you. You are one among many others.

I get a choice right now. I can see myself as a speaker who's standing in front of an audience saying some stuff, or I can see myself as one of this here we are together, a bunch of people who actually care deeply about what they're doing, and they're trying to advance it, and they get stuff done, and they feel strongly, and they have big ideas. What an amazing thing to be a part of!

It happens to be my turn in the conversation. It's a different view. It's a way to move your center of attention. It's an example, moving my attention outside myself to see that I'm part of something. Very powerful. However, there's an enemy to that, fear. It's the enemy of letting go, of letting go of the idea that it's about me, that it depends on me, that my view is the one that is the right view.

It's fearsome, because we attach our identity to that. "I get paid for my judgment. What are you talking about? Our team is expected to use its powers to accomplish things. If we let go, if I let go, who will we be? Who will I be if I let go?" The way to conquer this enemy is one more move of your center of attention. Recognize that the bubble's a bubble, and see that others have them, and see that they are a construct, that they can dissolve.

You shift from seeing yourselves as being responsible for getting it done to seeing yourselves as a collection of possibilities that are invited to participate in a still-larger story. It's not just us here. Why this conference this year and not three years ago? There is something happening in the world, yeah? This conversation wouldn't have happened in this way.

You wouldn't have all been here. You wouldn't have gotten permission to come. There's something going on, a gathering of possibilities, that we're a part of. Our job is to participate openly and wholeheartedly. That's the work, participate, get in the conversation. The conversation is the work.

There is some steps of opening, some degrees of opening that I find useful. This last thing, you can feel it in the room sometimes. Boy, a lot of times it doesn't happen. [laughs] Let's look at a picture of design capacity in an open culture. We send the researchers out, and in comes the orange. It encounters the culture, the identity, and the values, the relationships, the explanations, the dominant story.

People go, "Well, orange you say? Orange. Orange, but we do blue. It's orange, but we do blue. Let's work with that. Who does a blue company need to become in order to thrive in a world that has orange in it? Who do we need to become to participate with the orange? Let's explore the possibilities. We have a process for that."

Different. The result is that people are fulfilled. The outcome can be true, has a chance to be true to both the organization and the world, and both the world and the organization are changed by the process. That's the difference with opening.

Here's a way to tell whether design is getting into your culture. After every project, something about the organizational conversation has changed. There's a metric. You figure out how to measure it. I don't know. [laughs]

Here's a story. We got a call, this is three or four years ago, from a company that makes lots of stuff for people in their houses. We had worked with them before. We'd done field research. They said, "Hey, our quality function is a little leaky. Some stuff is getting through that kind of sucks.

Could you, because you're research people, come train? We've set aside four days, four and a half days, four full days, to train the people that make up our quality function so they can learn more from when they go out into home visits, they're already doing home visits, and better communicate what they learn."

Then, eventually in the call, "There may be some conflict. We just want to let you know." It was nice to have that warning. The first morning, we learned to move our attention outside our bubble. We learned to see without judgment, and we invited people to leave their jobs behind, leave them in the office. Then, we went into homes.

By the way, I should say who was there. Quality managers, pretty senior design people, usability people different than design people, engineers, field sales support, and eventually an executive or two, so pretty mixed. It's a cross-section. One of our rules of thumb is convene diversity and power the best you can. Convene diversity and power. These have a good mix of that.

They visited customers in their homes. Everybody visited at least two homes, and watch people use the stuff. That was the first day. Then the next day, we made them watch all the videos. They watched videos, and they captured notes on Stickies. It used a lot of Stickies. Think about what's happening.

Here, people who have been blaming each other for the leaks and the quality process. They're from different home cultures, engineers, design, and quality, and so on. Different languages. Instead of facing each other, they've all turned together to face customers, paying real attention. Then the next day, being forced to pay even [laughs] more attention.

Then, they did clustering. How do we make sense? Let's find themes. In groups, they clustered, and then they gave these great presentations. There was a side effect to this. It was not that it hit the culture and challenged the story. It was that, them paying that kind of attention brought them together in a way that...

[laughter]

I realized, I've learned from students that not everybody has this metaphor in their culture. The elephant in the room is the important thing that's big, that you're all afraid to talk about a name. I like this image better. This is from Banksy. She's so much does not want to see the elephant that she's covered it with the same wallpaper as her living room, just camouflaged it.

The elephant moment was that, we're digesting these people. We're talking about the consequences of it. They bound as a group. Now, they're one group, instead of four or five. There came a time, when a guy...his name is Randy...said, "Hey, if I say something, can it stay in this room?" He's starting to feel some safety. He got agreement.

He says, "I don't feel trusted. I don't feel trusted." Now, he's down on the bottom layer. He named a monster. "I am the guy who says red light, green light in the final tollgate. I'm the guy who says, go to production, or it's not ready. I feel that if I say red light, I will be blamed personally for costing the company millions of dollars."

He wasn't lying, making it up, he was in that position. He was [laughs] in that position alone, holding that red light switch. That broke thing's open. Through the conversation that followed, now these silo boundaries are dissolved. They're in it together, because they realized, "If he feels that way, how are we going to ship quality?"

What came out of that conversation, it got written up on a sheet is that, what we realize is we all care about quality. We actually all care equally that people get stuff that works, that they understand, that they like, that lasts. We all care about that. It happens to be our job. We have a job doing something we care about. Together, we change the agenda. In the last day of the workshop, they redesigned their quality function.

See, schedule is king. It was something worth trying. It wasn't like, "OK. This is how we want it to be." It was like, "What if we did it this way?" They called up an executive. In the last two hours, they pitched to this guy. He said, "I like it. You should try it. I'll fund it, and I'll give you time." That's a happy ending. Most of my stories don't have such a happy ending.

[laughter]

What happened? This company, they had tremendous design capacity. In this part of the organization, it was stuck in this repeating pattern. Other parts of the organization were closed to designs insights, because of lack of trust. People doing the design work were frustrated, because they didn't know how to persuade.

They thought persuasion is the only tool in their hand. "Can I say it to somebody else? Can I say it in a different way? Can I make different slides? How can I persuade them?" Getting outside together broke that pattern, which you remembered purpose down in their cultural layer. Then, they could have done it before. Then, they could design and experiment about reconfiguring their quality function. That's in service to the bigger story. The blame was gone.

This for me is like a tenet of culture work. Doing design together creates open culture. I won't say it creates open culture, it nudges culture towards openness. We said earlier, design capacity without openness produces laden results.

How do you open a culture, it's really difficult. It turns out that doing the fundamental creative process helps create openness. The process helps create the conditions it needs for its own improvement. By the way, your managers should know that if they say they're serious about developing design capacity, they're taking a virus into their organization that is going to attempt to pry open the culture.

It's going to try to get into conversations about identity, values, purpose, and relationships. It will try to raise that question, "Who must we become?" As supposed to how can we be a bigger me. Design is scary damn stuff, when you let it into your culture.

Since that workshop a big tip of the hat, this is almost, I say derived, I could say stolen. This form for organization becoming that efforts must be systemic. We need to convene a representation of the system. They have to do it together. They have to make together as to be participatory. It has to be emerged, and not figured it out, not to solve the problem. We have a terrible disease of solutionism. Holy crap!

Not figuring it out, but agreeing on a number of things worth trying now. One of the goals are on which we design our workshops in studios as partly taking lessons from that is get the other into the same room, and give people experiences that help them be abstract to the other. Help them act like shoemakers. There's some cool text.

We use a sequence that I will say to you, as a kind of building block. Document our bubble together. Just spill that out. Get it out. Look at the seven different ways on the wall seeing this thing. Then, immerse. Visit some slice of life, or each other's slice of life. You can fix this with the magic of...oh, that was a bad idea.

[laughter]

Then, immerse. Have an experience that challenges people's judgments and pre-suppositions, and give people time for reflection and little structure, because we're out of practice of taking time for reflection in a corporate environment. We use intuitive method, some like model making. We've had Central Pennsylvanian IT managers stay five minutes in silence. [laughs]

Then, facilitate dialogue in a way that makes it safe to say something, other than the old repeating story. You can stack these. You can use that as a design brief and stack sequences of these experiences together. It's powerful. My problem is that, there's too much to talk about. I had just pages lined up, maybe in a some future version there can be a workshop.

This is a conversation that a lot of people are having. A lot of it outside the corporate world, longer than the corporal's been having it. There's a lot of parts on the shelf. There's a work of advancing our own practice by putting these parts together in ways that make sense for our world, for our work. It's fun to show those to people, but not now.

There was a list for you. I want to conclude with a kind of invitation, because my favorite Irish poet with a corporate consulting practice, David Whyte, says that, "The game is up, as far as the illusion that you can live your life and run your business in isolation from a larger story. It's an illusion." We're all here together, because we're seeking creative change.

In some way, all of us here are trying to spark a lead creative change. It's one of the most dumbly difficult things you can do. It's personal, because it's hard. I've been studying, I would say, the poetry of this work, the poetry of culture work, the poetry of creative work. People have been seeking creative change for a millennia, some of them were poets.

Here is one thing they say. They say, "How do you know that you're on your path, because the path has disappeared? How do you know that you're really doing something radical, because you can't see where you're going? You may not even know where you are. You've traveled beyond the realm of planning. You're in the realm of uncertainty and self-doubt."

Dante, the Inferno, famously began his epic story of "The Human Journey" this way. This is the first lines, the first lines of the story of "The Human Journey." "In the middle of the road of my life, I awoke in a dark wood, where the true way was holy lost. In the middle of the road of my life, I awoke in a dark wood, where the true way was wholly lost."

That's what it feels like to begin. You're in the forest, and the way is lost. What do you do? This is a poem by David Wagoner, who is a Seattle poet. He made a whole book of poems that were drawn from Native American stories and teaching. This poem "Lost" comes from the advice of an elder to a young person, who would ask, "What do I do if I become lost in the woods, or lost in the forest?"

The Pacific Northwest. Think. Mettur, Cider. You don't know where you're, when you're 100 yards in. This is Lost. "Stand still. The trees ahead and the bushes beside you are not lost. Wherever you are is called here and you must treat it as a powerful stranger. Must ask permission to know it and be known. The forest breeze listen. It answers.

I have made this place around you. If you leave it, you may come back again saying, here. No two trees are the same to Raven. No two branches are the same to Wren. If what a tree or a bush does is lost on you, you are surely lost. Stand still. The forest knows where you are. You must let it find you."

Now, it's interesting. Like, here's attention again. You find the conversation by paying attention to your world, and your colleagues, and to your own interior, including the negatives and the fears, they're all part of what it is, part of the woods. It requires room for silence, standing still, to simply stand on what is and take it into yourself, and getting conversation with your own values and your own beliefs.

The attention then can flower into something else. Same goes for your team. The courageous conversations are often internal. The first ones are often internal, because we're out of the habit of standing still and listening. There's all these stuff waiting. [laughs] We're out of touch with our own cultural layer.

That does a much better job of noticing, than our busy monkey mind. I like reading "Lost" as a design poem, because if you read that way, it's clever, because it leaves out the making step. We get all the way to the end, and we're still [laughs] in the woods.

[laughter]

Wondering this advice from the elder. We're so apt to skip the steps of observe and reflect. We feel pressured to get walking. We feel pressured to get walking. I see design as a tool for moving through the forest, when you aren't sure you know where to begin. You have intention. You immerse. You listen. You reflect.

Now, you can take a step, without stepping on a snake and falling on a hole. It's design fundamentals for people to using companies. Open your attention, reflect, then walk. You can, by the way, use making. I won't say a lot about this, but we have iteration. We have this idea. I have the idea. Now, let me iterate. There is a way of getting through the woods.

I'm going to stop the woods metaphor for a second. I can try things. I can try a whole bunch of things, if I don't know what to do in a culture. I'm doing culture work. Some of them will attract people. Around those will grow a desirable behavior. If I thought ahead of time, I know how to stabilize that. Some of those things will be negative, or they won't become attractors at all. Great. I learned something. I'm going to try more. We walk through the woods. Making is definitely part of it.

What about the shoemaker and the strategist? We should end our fable, our poor arguing friends. To resolve their dispute, the shoemaker and the strategist began to go into the village together. They went to the schools. They went to the factories. They visited the market, and the offices, and the hospitals, and the gymnasiums. They visited the mayor.

They went to other villages. They spent time with other shoemakers and other business owners. They see that the world is changing. Other businesses have followed the examples of the department store. They also see that other shoemakers are clinging to their craft, finding a way to make a living off of shrinking demand.

Both the shoemaker and the strategist realized that they had been clinging to their own story. Reality is much bigger than either one of them had perceived. I can't tell you their ideas. They aren't sure themselves how it's all going to work out. But they're very excited, because they're in new conversations. They're asking new questions, questions full of creative process.

Like, they're asking, "Who are we together? Who are we together with the children and the teachers, with the blacksmith and the mayor? Who are we together with all the other small shoemakers and all the other villages?" Excited by these questions, the shoemaker and the strategist thought of 12 experiments that they can try. They know some will fail, but that's OK, because they don't cost much, and they don't take much time.

They feel certain that some will be exciting to the people in the village, or the other shop owners, or other villages in those experiments. Those experiments they'll do again, only bigger. They can't wait to see what happens. Neither can I. That is the end of the story. I'll stop there.

[applause]

UX Advantage: The Candidate Experience Is the Customer Experience

Lou Adler - The challenges we have ahead require top talent to execute. Design leaders, who could lead our organizations to new heights, are a rare gem to find and recruit. A designed approach to hiring will change the caliber of your team. From first contact through the candidate’s start date, the hiring process needs to be thoughtfully designed. You’ll see a performance-based talent system that replaces the talent repelling old-school HR processes.

Lou Adler: Jared was telling me last night, and he just mentioned it again today that you guys have some hiring challenges in hiring teams.

Can I just hear a few of those so I know what we're about here? Then I can tailor some of my remarks, at least, specific to those. What are some hiring challenges you're facing? Say them as loud as you can.
Audience Member: Retaining.
Lou: Say that again.
Audience Member: Retaining good people.
Lou: Retaining, because you're hiring people that don't work out. Are they good?
Audience: [inaudible 0:24] .
Lou: Got it.
Audience Member: We want co-location, but we're not located in the "hippest"...
Lou: So location's always a challenge for everybody. Keep on going. Some more?
Audience Member: Our department has no idea how to hire for UX.
Lou: No idea how to hire. Keep on going. Are you seeing enough good people? You seeing enough good people?
Audience: No.
Lou: Are you hiring people who are demotivated, possibly?
Audience: Yeah.

 [laughter]
Lou: Actually maybe the managers are. That was actually the humor part of this show...

 [laughter]

This could be along hour here, Jared. I've got the answer to all of your hiring challenges are in this little card here, which would be pretty remarkable if it's true.

I'm going to give you that card, but not quite yet. Your name is Alana? You're not sure huh? That's OK. I'm going to give you those cards, but don't hand them out yet. Just leave them there. Don't let me kick them.

What we're going to do is we're going to talk about hiring today. As Jared said, I wrote a book, and I talk about a lot of stuff. The essence of it is performance-based hiring. How do you actually hire people from a system, and a process standpoint?

Before I do that, I want to do something a little bit of...First of, if you have your hook up to the Internet, and you would like to get the handout for this, you can download it at budurl/uxoverview. I'll show that again, if you want to get the handout.

I want to do something a little bit unusual. I'm glad that chairs are here. Sometimes I sit down, and I say something pretty crazy. There was force. This could be nice comfort here. I'm going to say something a bit unusual.
 
I want to go fast forward to next year. How many of you going to be hopefully hire two, or three, or more people? So everybody. Let's say, a year from now, which would be August 19th, 2016, you've hired two, or three great, great people.

Because of that little card, you said, "Lou's card was great. It is unbelievable. I actually used it, and I hired two to three great people." Now, at the end I'm going to open up the Q&A. If you get the book, you hire five, or six great people. The card's still pretty good. You'll hire two to three.

 [laughter]

You send me on Slack, whatever Slack is an email. You say, "Hi, I just hired some great people. I want to thank you for the card." I say, "OK. Why were they great? What about them made them great?"

Think about the best people you've hired and the best people you will hire. You'll have to say a year from now, "These are great people." What evidence would you demonstrate that they're great? Just give me some ideas. Loudly as you can.
Audience: Self-motivated.
Lou: They're self...they don't need a lot of direction to get it done. Keep on going.
Audience Member: Fit.
Lou: They're what?
Audience Member: They fit.
Lou: They fit with the team. They work well. Great. Keep on going. Is that what you meant?
Audience Member: Willing to learn.
Lou: Say it again?
Audience Member: Willing to learn.
Lou: Willing to learn. Great. And they have learned. Not only willing. They could be willing to learn and be stupid, but they're willing to learn and they learn and they apply, right? Sorry for that. Hope I didn't offend anybody on that last remark. If I did, you wouldn't know it anyway.

 [laughter]

This is actually...well, it must be Baltimore. Keep on going. Great person. Why are people great?
Audience Member: They work hard.
Audience Member: They actually have talent.
Lou: They have talent. They work hard.
Audience Member: They work well with a team.
Lou: Work well with a team.
Audience Member: Dependable.
Lou: Dependable. You say it. They're going to get it done. Here's the thing. I've asked this question to a lot of people, engineers, market people, salespeople, executives. Here's the top traits of top people. If I miss some, put them down.

Consistent, high quality results. Not just once. They always get high quality results. In terms of design, whatever they do, they do great design. Whatever needs done, they do it well. They coach/manage themselves and they help others. They mentor others. They can work well with a team.

They have what I call "leadership." In my mind, the definition of leadership is a vision and the ability to execute on that vision, whether it's big or small, but, "Hey, here's what I think we should do." Then they go do it. They don't just talk about it.

They can figure out a problem and they come up with a resolution they actually implement. They're flexible. They can deal with change. "Hey, we got to change this. We're doing this."

They get it done. "Hey, yes. I need it done." "You need it done next week? Fine." They'll get it done, and they exceed expectations. There's probably other things that great people do, but do you think that's a pretty fair list?

Well, I'm going to contend that every one of those things is predictable before you hire the person. The answer to do that is in this little card. I'm going to walk you through it.

But just think about self-motivated and directed. First off, you have to be able to predict the people we hire. All of those mistakes you've made, if you've followed it, you knew those were going to happen.

You don't want random results. They happen all the time, because we're using a process that doesn't predict quality of hire. So you think about it. Part of it is in every one of those cases, you've defined the work. The work, they exceeded expectations.

They did the work on time. They did more than they're required to do. They dealt with change, but it's always about the work. If you don't define the work before you hire the person, it's random luck if you'll hire the right person.

By understanding if, "Hey, this is the work I need done. Let's see if we can get comparable work that you've done in the past in a comparable situation."
 
If they're growth rate...if you'll get their track record of performance over time, if it's like this, you've probably got a good person. If it's like this, you don't know. If it's like that, you've got a bad person.

Just by looking at their work history and see the size of the projects, the teams they worked with, you can start saying, "Hey, this is probably a pretty good person."

I call it the "achiever pattern." You can look at any person, and look at their track record, and see if they're in the top 25 percent of their peer group. In sales, people make quota time after time, year after year after year. In engineering, they win awards, they have patents, they give white papers, they speak to groups.

There's things that people do that you can forensically look at their background and say, "This is a good person." If you tend to always hire top 25 percent people, they'll probably wind up being a top 25 percent person.
 
They fit with the manager. This is as critical as anyone. If you're the hiring manager, and you don't work well with that person, it's bad for the person. The reality of the managerial fit is as important as the ability to do the work.

Also, from the candidates perspective. This [inaudible 6:42] talk, I want to talk about the candidate experience which is critical. Candidates want to know the work they want to do before they do it.

If a candidate says they're not sure what the work is when they accept the offer, I would think that's problematic if the person will be successful.
 
"Fit" is this person tapping into that candidates' intrinsic motivators. Certain people like to do work with teams, certain people don't. You got to tap into that and figure that out ahead of time. Does the candidate see that as a career move in comparison to everything else he or she is looking at?

This is what I want to do. On this card, you can assess all of those things. That's what we're going to do today. Here's what I call, "The five pillars of an exceptional candidate experience."

My background, some of you know it. I'm pretty old. I've been in the recruiting industry since 1978. I actually worked for a living prior to that for 10 years, so you can tell pretty quickly how old I am. Was running a manufacturing company and I literally been in thousands of search assignments.

We're working with companies now around the world, helping them design hiring processes. When we talk to candidates, here's what the decision to hire or not...how it affects their decision.

Number one, and from a company standpoint, do you have the right strategy in place to go after top talent? When I look at 9 out of 10 companies, they have the wrong strategy. They have a "weed out the weak" strategy, not "attract the best" strategy.

Number two, the job itself. Is this a job that a candidate can clearly say, "Yes. That's a career move"? The first moment the person hears about it, they have to make that decision.

The third one is the recruiter. How many of you guys use recruiters to find people? I want to see that again, because it was only about a third. How many of you use recruiters?

The recruiter is your sales rep for that job. If he or she doesn't know the job, you're missing on a lot of good people. We train recruiters and hire management in the world, the recruiter is the sales rep. If he or she doesn't know the job, you're done for.

You, the hiring manager. You think, and there's a motto here, or an adage, that says, "Hiring managers hire in their own image." The truth is, the best candidates accept jobs in their own image. If you're not a leader, you're not a mentor, you don't know the job, they're going to dis you pretty quickly.

The process. Is the process professional from the moment of the first contact to the final close? All of those things are critical. We're going to have time to talk about those at a pretty much highlight level. Again, there's the hand out if you want it. [inaudible 9:01] url.com/uxoverview.
 
If you want to reach me, I think it's probably on the card here somewhere, info@louadlergroup.

Just read this, Jared knows about this. I was contacted about five months ago from a professor at Harvard. He sent this very nice email to me, and we're doing work with Harvard right now, so just take a look at that, where we're going here.

The problem is, I didn't tell him I could summarize it all on this little card. He would have taken the email back. Now let's talk about the right strategy. When you think about hiring, there's only four big decisions that are typically made. Everything you do can be divided into one of four buckets.

First off, let me just ask you this question as I get into that. Is there a surplus of great talent for your jobs, or is there a scarcity? How many of you would say scarcity? Surplus anybody? Hard to find surplus. Let me just make this statement, and then I'll prove it.

You cannot use a surplus of talent strategy in a scarcity-of situation world. I'm going to contend most companies, and I've looked at a lot of your job descriptions, you believe there's a surplus. There isn't a surplus. There is a scarcity.

When you think about the hiring process, there's four big decisions. The "Having" decision is a person supposedly has to have on his or her skill set and resume. Skills, experience, academic background, levels of this, and competencies, and all this stuff.

"Getting" is what a candidate gets on day one. They get a job title, they get a compensation, they get a location, and they get a company name. "Doing" is what they're going to do in year one. It's the actual work itself. "Becoming" is what they can do if they do the work well.

Most companies build a strategy from left to right, weed out the weak. "Let's post a boring job description, let's let everybody see it, and let's see if we can get good people." It doesn't work.

There's not one great person that says, "Yes, I'm going to get another year of doing exactly the same work I've done before, get another year of skills." Nobody thinks that way, but our systems are designed that way.
 
A scarcity situation, you have to emphasize what people can do and become before you worry about the skills. I guarantee that if they can do the work they have all the skills they need, and it's not what you've written on your job description.

I'll say that again, if they can do the work, they have exactly the skills needed, in exactly the right balance. I have to prove that, but I'll say it. If you want to hire great people, you've got to emphasize what they can do and become. Not what they have and what they get.

This requires, I call this the double shift. You the hiring managers have to define the work, not the skills needed to do the work. Recruiters have to tell the candidates, "Hey, don't worry about the money, lets tell you the career move first." If your recruiters can't make that pitch, you're not seeing 90 percent of the candidates you should be seeing.

You recruiters will determine the success of your hiring capability if you use recruiters, but it starts by you taking responsibility for telling the recruiter what he or she needs to know. The idea is, we're going to go from a right to left strategy, attract the best, not weed out the weak, that's step one.

If I just asked you, what do you think is the strategy you're using? A weed out the weak or attract the best? First let me ask you, how many think you have a strategy that's to attract the best, just to give a sense? Three. How many of you have weed out the weak? Four. There's 500 or 300 of you.

The point is, if you post boring job descriptions that emphasize skills and experiences, using a surplus strategy, which will not work in the scarcity of talent in our world. It's also demeaning. The other word must have you got the wrong strategy. I had certainly a terrible candidate experience.

Let me show you some other work here that I want to emphasize. I do a lot of work with LinkedIn, we did a big survey, and I'm going to be talking to a number of people next week on the results of the survey. How do the best people get jobs?

I've done some other surveys, but the essence of it is, and here's a chart we did. It's subtracted down, but nonetheless, it's almost 3,000 people. The blue line and the orange line are the most important. The graph on the far left, on the chart in the far left says, "Underemployed, or unemployed?"

We asked candidates, "How did you get your last job, and when you go it, were you active, were you kind of semi-active, or were you passive?" On the far left were active candidates who were underemployed, almost two times as many people got their jobs via network.

The orange is network, and the blues applied through a job posting. The second line from the left says, "Employed and active," so they're fully employed, but active almost a little bit more people got their jobs via networking. Even for active candidates, 60 to 70 percent of the active candidates got their job via networking.

I'm going to say there's a pretty small pool of active candidates. Passive candidates, the ones in the middle in the third column over from the left, these are people who've started to think about leaving, but just tiptoed into the market. Four to one got their jobs via some employee referral, or some networking, or some person they knew.

A passive candidate is 20 to 1 networking. When I talked to companies, most people spend 80 percent of their time on job postings, and wonder they can't find enough good people. Which is not only a bad strategy, it's also a bad approach to finding these people, because you're using the wrong tools.

I'll make it even worse. Only 10 to 15 percent of the total market is active. That's not a lot of people, because there's 85 percent that isn't active. 15 to 20 percent have started to look, and 65 to 75 percent are passive.

You're sharing 10 to 15 percent of the total market with everybody else who's competing for the same people. Indeed there was 31,000 new UX and UI design jobs. Probably more than that, but that's what I found, and I'll show you a couple of postings.

The idea is, if people get their jobs through networking, then you've got to spend most of your time leveraging your employee referral program or your networking program, or using recruiters who are highly networked.
 
Just start thinking about laying the landscape, you have a strategy that has to focus on what they can do and become, and a sourcing approach on how to get jobs via networking.

Let me kind of define it. I did this with a huge company yesterday in New York City, they said they're not seeing enough good people. I said, well, you see this little x, which is the 5 or 10 percent of the people you want to hire, and you're sharing that with 50, or 30, or 100 other companies that also want to find that person.

These people are skills and experience qualified. They have all the skills you want and they're applying, not necessarily the definition of a great person, a person who's great who has all the skills, wants to do something other than what you've written there.

Now, there's another group of people that's at least 10 times bigger than that, who saw the posting, or didn't see the posting. They actually are skills and experience qualified, but the job posting was so boring and cumbersome, so hard to apply, they didn't even want to engage themselves.
 
That's 10 times, but now there's a group that's 20 to 30 times bigger than that x, who are performance qualified. I talked about the strategy. They can actually do the work, but they have a different mix of skills and experiences. They are active candidates who actually if they saw it would say, "That's a pretty good job, maybe I'll take that."

There's an even bigger market, 50 to 100 times that are performance qualified, but they're passive, and they need to be contacted and nurtured. You're just seeing such a fraction of the market. Many of your problems are such that you're just not seeing enough good people.

Even if you were seeing them, the jobs that you're posting are boring, so it wouldn't matter if you saw them. There's a lot of overhauling that needs to be done here.

One overhauling is that, you can't use or find your posting in a haystack for candidates, you apply in some boring demeaning process, and you weed out the weak, which is a surplus strategy.

To get the best people, you've got to reach out to them and nurture them, and it takes time. The hiring manager is a critical orchestrator of this time process. Finding people who apply is very transactional. Finding a great person, and the outer rings are there, is consultative recruiting.
 
You have to nurture the person, you have to actually say, "This is an interesting job, would you be open to talk about it?" Different process, the job itself you have posted, now I'm going to show you a few in a moment here.

They're pretty bad. They're ill-defined, they're at best lateral transfers, versus a career move. One of the criteria of a great person is, they have to see it as a possible career move.

They have to see about it in the writings that you put out there. They have to hear about it from the recruiter, and they have to hear about it when they have a conversation with the hiring manager in the first time. If it doesn't clearly that, they'll opt out in droves.

The final piece is, we're focusing on quality of hire on return on investment, not cost and efficiency. I talked of HR leaders around the world, heads of talent, and, "I've got to reduce the time to fill, I've got to reduce the cost to hire." They never talk about, "I've got to improve the quality of the people I'm seeing."

I don't know who's in charge. I say, "Well, whose in charge of quality of hire?" They say the hiring manager. I talk to the hiring manager, "No, it's HR." To my mind, the best hiring managers demand great people, and they'll go out of their way to get it.

So I'll put the burden on the hiring manager. That's the person who should be responsible. That's his or her team. The idea is, you're only seeing a fraction in the market because you got the wrong strategy. The market is, 95 percent of the market, or 99 percent of the market, is outside of what you're doing today.

One way to solve the problem that you've identified earlier is to expand the pool, but to expand the pool, you have to change your process and the strategy as well.

Let's understand why people take jobs. This is very important, so you can get a sense of, from a macro level to a little bit more micro level. I've asked recruiters and done surveys around the country, and I ask recruiters, "When you first talk to a candidate, first conversation, what do they want to know?"

The criteria during that, what do they want to know before...? You get a name of a good person and they say, you call a person up and what do they want to know? They always want to know what they get on day one.

They get a title, they get a compensation package, they get a company name and they get a location. Not interested, not interested, not interested. With most people, you'd think that would be logical.

However, this is another survey with LinkedIn. This is actually the survey I'm going to be doing next week, but I've known it for many years.
 
When you ask the same people three or four weeks later when they actually get the job, who have they really a strong person, "Why did you accept this job in comparison to everything else you looked at?" Totally different.

It's the career opportunity, the job and the impact. The hiring manager and the team they're going to work with is critical to the decision. The compensation and work-life balance is there, but it's not number one on the list, it's fourth or fifth of the list. As long as you're above the threshold, that's not important.

As long as that's the fare, if you're really below, then it a problem. But as long as you're above the threshold, then the work itself and the team they're working with is much more important than the compensation, even the location.

They also want to be invested in the company culture and the mission of the company, because they say, "Business and this an industry I want to be part of." That's the criteria. But if you screen people on the criteria to engage, you're going to lose them. That's day one criteria. That's again going back to the surplus mentality.

Recruiters say, "OK, you fit with my budget. They have the right skills. Let's go forward." That's not how great people make the decision. They make the decision based on what they're going to do. At every step, you have to focus on the career opportunity and the career move.
 
Let me just give you one sense if I had a class of recruiters here, because at this point in time, it's, "Well, what do I do?" They say, "What's the money?" Every good recruiter wants to know that, because 99 percent they [inaudible 20:51] , "What's the compensation?"

I do a lot of work in Silicon Valley and I'm moving around. I told recruiters, "Is this what you say? If the job doesn't represent a career move, it doesn't matter what we pay you. Let's first see if it's a career move, then we'll figure out if we can work the pay."

The idea is when you have to nurture person, it takes hours to figure out the criteria to accept over an extended period of time. Two to three weeks. "Let's just talk about it. Let's nurture." They got to understand that they've got to convince themselves, but what if they're not looking?

You call a passive candidate at 10 o'clock in the morning. At eight o'clock when they left for the office, they weren't looking for a job. The recruiter gets in touch with that person 10 or 11 o'clock, that person goes home at 9 and says, "I got a call from a recruiter."

The spouse or significant other, best friend, says, "I didn't know you were looking for a job." "I wasn't, but it sounds like an interesting job." The best friend says, "What does it pay?" The candidate then's got to say, "It's not about the compensation, it's about the career move. This looks interesting."

Plus candidates don't just sell themselves, they got to sell every person they know. Recruiting a passive candidate is different. Totally different. It involves a hiring manager, it involves recruiter, it involves the hiring team, collectively. That's the experience. It is a different experience.

You have a transactional process, You're not going to hire great people. All the problems you said, in my opinion, are result of a transactional hiring process, not a consultative hiring process.

Now, you're tracking the right audience. My wife said I should not put this picture of this new Miata up there. But I saw this Miata at an art show when where I lived in Laguna Beach, and I said, "I had to have that car." My wife says, it's a new Miata, says, "You're too old. You don't need a car."

Then I'm reading about this Miata. It's designed for 40-year-old males, who have $30,000 and no kids that they can spend, just because they want a new car. I told to my wife, "You're still too old for the car." [laughs]
 
I just looked at that, and I didn't know that was designed for a male, 40 years old, whose kids are leaving, and had a little $30,000 extra to buy a car he didn't need. I said, "That's me. I want that car. I still want it." I was thinking about, "How can I talk to my wife, and let me have that car?" I cannot figure out how to do that yet, but I'm still thinking about it.

The point of that is, it was designed for the right person advertising. It's just a car. Just hit me. But now what's your target audience? Does your advertising, and we all talk about the design. Advertising is part of design. Does it meet your target audience?

This is how your job postings look on Indeed. One of the primary sources of where active candidate see it. I just put in the words, "UI," or "UX," 31,000 job postings. They're boring.

Look at this Senior UX Digital Design. Read these words. This is the words. Number one, "Senior UX Designer, Mobile Apps." That's hot. "UX Designer for..."

 [laughter]

"UX Designer. Four plus years of experience as a professional concept designer. Will build the life full polished products that excite." Number two. "The successful will be able to build strong relations across the organization. Apply for UX Team now." You should be embarrassed.
 
[laughter]
 
But now I pick the few. If you think this is bad, here's what the best I could find said. The best, and it was pretty bad. "User Experience Designer. San Mateo, California." This is the hottest market in the country, just north of San Jose.

"Marketo is the world's most loved marketing software platform. It's loved in large part because of the experience." Then it says, "What does Marketo do?" They don't tell you. You have to go to another website.

 [laughter]

"What do Marketo UX designers do? Here's an example." You got to go to another website. "What makes a good Marketo UX designer? Creative." This isn't very creative, you come up with new solutions.

OCD. Here's interesting. This was actually interesting, but it was more for an English major. "OCD. Does it bother you that there's a misspelling in this question?" Don't worry it's there.

 [laughter]

Then that number four was concise. I'm sure the person who wrote that said, "Oh, that's so funny. They'll laugh about being concise."

Here's a more typical one. It has all the list from the ATS system, which is how you post jobs through the Applicant Tracking System. None of these stuff matters.

Why can't some designers get into the idea system? No candidate cares about the location of the title, or the administrative stuff that's on top of the design. This is a terrible design in and of itself.

Then is says, "Mandatory technical skills." That's really appealing to somebody who's looking for a career move. Yet, what it says is actually pretty interesting. It says, "Discuss projects with clients to understand their business objective, scope and plan, understand..."

That's actually not a technical skill, that's actually the work. In some way, it's OK. It's not great, but it's OK. I'll give you a great one. But to me, this is how you start. That's why you're only getting one percent, you're not seeing very many good people.

I always suggest is have a great posting, even though the 99 percent aren't going to go, that you can drive people to the posting. It is critical you have a great posting. I'll give you an example of an email that can be converted to a posting.

Now, I'm talking about person descriptions versus job descriptions. I looked at about 30 of those and they were all terrible.

It doesn't surprise me you're not hiring good people. A job description with skills, at least what's published, skills experience, academic background. [inaudible 26:25] comes in, says, "What are you looking for?" A recruiter asks you, you take the assignment.

This is what you put, "I will need, seven years of this, two years of this, his skills got to be great at this, great at that. His competencies, because you have a competency model, and a better responsibilities."
 
But when you think about other than responsibilities, that is not a job description. I'll say it again. That is not a job description. It is a person description. A job doesn't have skills. A job doesn't have academic background. A job doesn't have competencies. A person has those things.
 
When you think about, let me just ask you this. Have you ever met someone that has all of those of those skills and competencies who is not a top performer? Have you ever met somebody? Has all the skills and competencies. He's [inaudible 27:10] , right?

Have you ever met someone who is a top performer who doesn't have exactly that background? Have you ever met someone who is a top performer? I would tell you a hundred percent of every top person doesn't have that.

The definition of a top person is getting more done with less. You get assigned bigger projects in the first year. You get promoted more quickly. By definition, the best people have less experience. They just do more with their less experience.

If you're a top person, you say, "Yes! I want to do exactly what I've been doing, and beat the meat in the process." That's really what you're saying. The candidate experience for people who apply is miserable. That's why smart candidates understand how to network, which is I teach for recruiters how to get names of people via networking.

When I ask a hiring manager, I said, "Let's put this person description in the parking lot. 'What does the person need to do to be successful?'" This is the question I asked you earlier. I said, "You've hired people, and on August 19th, 2016, you sent me an email and thanked me."
 
I ask, "What did these people do?" You described it. "Why would you think these people are great people? It's because the work they do, so let's define the work they need to do to be successful." That' not rocket science, it's common sense.

A job description is a job. It's things that people do. They grow sales. They launch new products. They build teams. They evaluate processes. They do stuff. It all starts with an action verb, not a passive, "Have to be responsible for...Mandatory."

It's passive this, and if I want to do their work, that's a good move. "I want to grow sales by 10 percent in northeast territory. I want to build a team of four accountants to launch a new international [inaudible 28:51] practice [inaudible 28:52] consolidation project." Then people want to do that stuff.

I guarantee, if they can do that work and hold that as a standard. The ability to do that work and be motivated to do that work, they will have exactly the skills needed. Exactly. They couldn't do the work if they didn't.
 
You don't have to say the work. You say, "Hey, can you do this work? Let's prove it. Prove it to me you can do the work and have the skills." Here's the difference maker to match my quality of hire.

Focus on what people need to do with what they have, not what they have. Define the outcomes. The outcomes are direct. They determine the skills needed. The best people need less skills.

I talked to a director of engineering of a big high tech firm, had to have 7 to 10 years for this new the little chip design, a high-powered chip, it was a few years ago, had to have 7 to 10 year to had to have a senior designer.

I said, "What if I found somebody who had three to four years who could do that work. Would you want to see him?" I said, "An exceptional person who could do, would you want to see him?" "Absolutely, I'd want to see that person. It means he's an all star." I said, "But you never will, because this says you have to have 8 to 10 years."

I said, "Define this result. You got to design a state of the art circuit, I can accept that. But don't put in these qualifications that are meaningless or demeaning. Focus on the work, what they can do and become as a result of that."

I ask these people, I've asked this question literally from 1990 to today to 30,000 hiring managers. This question. "Who would you rather hire, someone who can deliver the results, or someone has all the skills?"
 
How many for results? I got to have 105. How many for skills? I'm not adding any. Did you just raise your hand? [laughs] It's a mistake. It's all right, I'll subtract that one from that, it's 104.

The idea is this is performance qualified. That opens up the pool 10 times the skills qualified. It all sums the pool. The diverse candidates, returning military vets, just the total change in focus.

If they can do the work and are motivated to do the work, you got a hot candidate. Why would you exclude that group by using a surplus model, which is having, getting, doing, becoming that surplus? Let's focus on what they can do and become, and screen them on that.

A couple of companies I knew I guess are coming up with different ways of screening people based on what they can do and become. But the idea is, it's a heck of a lot more exciting to focus on the work itself.

Let me ask you this. If you use recruiters or you've ever been approached by a recruiter, all right let's talk about it from a consumer's perspective.

How do you feel about a sales rep who approaches you or tries to sell you or your company something, or even in a retail environment, where the sales rep doesn't really know the product? They're just trying to hustle for the sale. How do you feel about that? Pretty bad, right? You just want to walk out.

That's how candidates feel about recruiters, who don't know the job. They're just trying to hustle to feel. It's your responsibility to make sure your recruiters understand what that job is.

You, as the hiring manager, have to say, "Yes. I will see 100 percent that people can do that work, but you have to prove to me they can do that work."

I talked to, was it Bill? Bill, are you here? You're here, I know. I saw you. It was Bill from PayPal. We spoke yesterday. He told me he embeds his recruiters to his department, which is a critical component. Recruiters have to recognize.

The hiring manager have to know the people, have to understand that not only the job, but also the experience. Focus on performance quality. It opens the pool from 10X-100X. Then, I always ask this question.

Why would a top person want this job, if it weren't for the money? What's the employee value proposition? Why is this a great job for someone who's not looking, and isn't going to take the job, other than money?
 
If you can't answer that question, you will not hire a great person. Just post your boring job and get the best you can. Because if you can't answer that question, you'll never hire a great person. When I take assignment, I just say, "I can't do it."

If you can't answer that question, you're affecting a person's life here. If you can't answer the question, you're not going to hire a great person. Not only defining the work, but then, why is this a great job for the right person?

Now, you probably can't read this, so let me read it for you. I don't think you can. I'm going to read this email. I was talking with somebody, who was Dana. You're from Michigan, right Dana?
Dana: Yup.
Lou: Remember I mentioned I had a company in Michigan. It was in Kalamazoo. I was talking with Division President needed to VPHR. I was trying to think, "How can I get anybody to go to Kalamazoo?"

I can't believe I did it, but this email helped. It says, "Subject letter, an open letter from the CEO to my next VPHR." Jane. That wasn't necessarily Jane. It was Marge, the right name. [laughs] "I need help making our business become as big as it possibly can, but we can't do it without you. Here's some background."

"Our company is growing very rapidly and more stretched our current nature policies, procedures, and town's acquisitions program to the breaking point."

"We need to become totally focused on what really matters, every single person in this company, and everyone we'll be hiring in the future. If we get this part right, there's no stopping us."

"To put off though, we need topnotch VPHR, and to build the right type of HR organization and infrastructure to address this unusual global opportunity. Without the right HR executive, we will not be successful."

It goes on to say, "Hey, not only we have a seat at the strategic table, you also lead it." I found 60 or 70 people on LinkedIn. We're not looking 40 percent, one that talked to me about this job, 40 of the 60. Didn't even list a single skill.

Not one skill is listed there. Big objective is turn around a company, from the HR perspective. Capture their intrinsic motivator that turns something big. The way we said it is, "If you're interested, send us a half page write-up of something you've done that's most related."

40 out of 60 people sent little summary of what they did, which I thought was remarkable. For advertising purposes, tell stories. Don't list a bunch of boring jobs. You don't need to list skills. There's no law that says you need to list skills.

A person can take the number one labor attorney in the country. David Goldstein, Luke Mendelssohn. You do not need to list skills. You just have to have something that's objective. This person needs to turn around a large $100-million organization. It's objective.

Capture the intrinsic motivator in the story. Emphasize what they going to do, what they going to learn and become, the doing becoming the future. Attract the best, not weed out the weak, and sell the discussion, not the job. It's not transactional. It's consultative.

I told recruiters, "Don't sell the job. Sell the conversation about the job, about the career remove. Sell the next step, not the fill the job." Too many recruiters were in this transactional mode. "I got to do this quickly. I got to do this quickly."

No great people are going to be hired quickly. This is where I'm going to do something a little bit unusual. I have time to do this too. One of the most important question I ask is, once I understand the job, we have to launch a new product line. You're responsible for this part of it.
 
Whatever the task is, I ask the candidate to describe something they have accomplished, that's most comparable to that. Obviously, I don't have an open job for each of you here. I have 110 or 120 people. I'm going to ask my more generic version of the most significant accomplishment question.

You could still ask this question. This question is, think about your most significant career accomplishment, the best thing you've ever done in your career. If we have 15 minutes to talk about it, what would you like to talk about?

The way I set the scenario up, I'm going to interview everybody now on that question. I'd like you to think. Imagine that you're coming into my office. We're going to talk for 15 minutes only.

The question will be is, I want to hear for 15 minutes your most significant career accomplishment. You got 15 minutes to tell me until it's the end of the interview. This should be a career defining moment.

Something that if somebody understands that, they get a good sense of who you are as a person. Take 20 seconds to think about what you want to talk about, and I'm going to go through here. I'm going to get ready to interview everybody in the room at the same time. This has never been done in a public group.

 [laughter]

That's why I need to sit down. These chairs are so sleepy though. I don't know if I'll make it. This one, I'll take 15 minutes. I'll take three or four minutes to go through it. I'm going to ask you a series of questions. Do not write down anything.

I urge you just to think about how you would answer the questions, if you are actually being interviewed this way. You'll learn it better that way. This is the card I'm going to give you. It's on here. You just have to follow the rules here. These are valuable card here.

You come into my office. I say, "Hey, very nice to meet you. Why don't you give me a quick overview of your most significant accomplishment? You can give me 30 seconds, 1 minute, 1.5 minute overview.

If you talk more than two minutes, I'll shut you off, but now I get the essence of it. I want you to give me a sense of...first off, how did you get this assignment? How did you get it? Did you volunteer for it? If you volunteered, why did you volunteer?

If someone assigned you this project, why did they assign you the project? Let's go to the beginning. In fact, when did it actually take place? Give me a sense of the month and the year just so I get a feel for it.

Give me a snapshot of it. What are the big challenges you face right away, when they got there at the headline level? Let's go fast forward to the end. By the way, how long did it take to complete this? Let's go fast forward to the end.

Give me a snapshot of the end. What are the big changes you made? Snapshot at the end. Let's go back to it. If you are to pick the one or two biggest challenges to overcome that, what were they?

One or two biggest challenges. Got it. OK, good. In that channel, what was the environment like? When I mean the environment, what was your manager like? What were the resources you have? What was the pays?

How do people make decisions? Is that an environment you like, or is that one you just live with? I assume you had a bunch of skills you brought into this job. How do you apply those skills? What were they? And how did you apply them on the job?

Did you learn any skills? How'd you learn those skills, and how'd you play those on the job? Let's get into initial. Where did you go the extra mile? Give me some examples of we did something we didn't require to do. You just went overboard with it.

I want you to give me two, three examples of we took the initiative. What was your single biggest success? Walk me through that. Why do think it was a big success? What's your biggest failure?

You say, "I wish I hadn't done that." I assume you put a plan and place for this. Walk me through how you did the plan. How did you build the plan? How did you get the resources of your plan? How did you manage to the plan?

By the way, did you achieve the plan? Was there any time you were going off plan, you had to recover? Walk me through how you did that. What was the single, biggest problem you had to face? How'd you overcome it? How'd you address it?

What was the single, biggest decision you had to make? How'd you make that decision? How'd you make the trade-off analysis? Why did you choose that way? In your mind, in retrospect, was it the right decision? We're getting close to the end here.

If you could do this whole project over again now today, versus X years ago, would you do it any differently? What have you learned? Would you learn about yourself? I assume you grew as a result of this significant career challenge.

How did you change as a result of this? As you think back in retrospect, what did you like most about it? What you really you inflow? You're highly motivated. Would you like lease, so you just got it out or delegate it? Time is up.

Wait, wait. I got one last question. What's your significant career accomplishment? What kind of recognition did you receive for it? In your mind, was the recognition appropriate for what you did? Thank you very much. Hold off handing this out. I just want to ask a few questions.

Given that, would I have learned anything about you? Not one question. Probably would. The idea is, if we have a smaller group, we have people write what they've learned about you on that.

They say tenacious, good technical skills, team project orient, what motivate...you could see it's a bunch of behaviors and competencies they write. Great at problem solving, whatever it is.

I'd say, "I didn't ask you a behavioral question. I asked you an accomplishment-based question, where you had to provide all your skills and competencies, collectively to accomplish some task."

People realize that's cool. I really learned a lot. Not just the form of the question I can ask, whether there are different ways of asking the same question. Typically, a lot of behaviors and skills standout as a result of this.

There's a lot of times managers, "Well, the candidate didn't tell me that." I said, "It's not their responsibility to tell you. It's your responsibility to get the information." If you are judging the person on his or her own presentation, you're not judging performance.

You're measuring their personality and presentation skills, not their ability to do the work you want done. You got to dig deep, to get this information. Let me ask you this question from a user experience standpoint. If someone interviewed you that way, how would you feel?

Would you feel good? How many of you would felt positive about this kind of interview? How many of you would have felt a little bit concerned or uncomfortable? You feel free.

Some people who have a few accomplishments, I tend to give people the accomplishment question ahead of time. Thing about it that I want to know what they've done. I'm not trying to trick them. I really want to know what they've done.

When I listen to candidates who said that, when a manger asks these questions, even when I don't get that job, that manager knows who I am. They know when I'm capable of doing. "I want to work for that person."

For managers who start selling right away, "I don't want to work that. They have low standards, or just trying to hustle me into a job." The idea of a deliberate, structured, in-depth interview, makes candidates feel better.

They feel, "Hey, if everyone else is hired this way, this is obviously probably a good team." This is a professional interview, and it needs to be done. You have to understand. Even if you know the candidates got used you got to ask it. I spend 10 or 15 minutes on this.

I also ask it multiple times. I ask similar questions for every job, so I start seeing a trend line of performance overtime. It does start the recruiting process. What I do is, I look for gaps between my job. Am I able to find my work? I know what work it is.

I built a performance-based job description. What I'm looking for is, this is the candidate. Here's my job. There's actually some stretch factors here. I look for, is there a difference of two biggest stretch?

If the candidate's too light, if there's no stretch, the candidate's too heavy. Put 10 or 15 percent stretch that has a good job. That's how I can demonstrate that the job's a career move. The interview is really key part of that.

Fact finding, peeling the onion is the key. Let me do this. Do you think this card would be helpful? It also says how to assess the candidate. It also shows how to define the work. Summarize my whole life 30 years of work in this little card, which is depressing, but I did.

 [laughter]

I got the idea from Baja Fresh. They have little three pork menu. I said, "That's cool."

 [laughter]

We've got cards. We'll give you each one. Remember, if you use that, a year from now, you're going to call me. Give me one minute to hand those out. Jared, I thought you had a fast group here?

 [laughter]

We have a performance issue here.

 [laughter]

Does everyone have one? You get one. Very deliberate group here. Here they're coming. Let me talk about the process a little bit, and then we're going to open up for some questions.

When you think about the hiring process, there really is a funnel. When you look at the total market, if you just use the people who apply, you're seeing less than one percent of the total market.

You tend not seeing the best one percent of the total market. What typically happens is, you have to build a pipeline or a list of names. You got to contact these people. You got to nurture these people. Hiring managers even need to engage with people.

A lot of it involves networking, leveraging employee refill programming, building out your networks. You got to recruit these people, then you get them to become serious candidates. Interview them, and you hire them.

It's a lengthy subject, process. What typically happens is, most companies have with this active process, they filter people through this having and getting screened. Eliminate the best people is the result. They still want to do that faster.

The guys, "We're doing it real fast." Doing the wrong things faster isn't necessarily a good thing to do. A scare strategy means you have to do all of the steps, all of the steps. That's what performance-based hiring is. In fact, I was with a company that has tens and hundreds and millions of dollars.

They're redesigning a new job board. They invited a group of people up to San Jose area for a full day of totally redesigning the job board, which was cool. I was really excited about it.

It turned out they had three or four ideas. What they're already doing, they just asked us if we like that idea or not, which I thought was disappointing. One guy, next to the designers in the room.

One guy said he had this cool design for the apply button, really fancy. I said, "What do you think of this design? It was a real UXA designer." I said, "I don't like it." I said I would design a button with four rings."

I just drew this chart. The outer ring is explore. Don't make people apply, because 99 percent of people don't want to apply. They want to just explore and have a conversation with the recruiter or hiring manager.
 
Maybe even a video conversation, or just even a video from the hiring manager. "This sounds interesting." That's the first step. The second step is, "Hey, would you seriously consider this?" The first step might be 5 or 10 minutes. The second step might be a couple of days. Let's have a serious conversation.

This could be a cruel ponder at, little bit more in-depth. The next one might be, and I forced this issue is, I want all candidates to meet the hiring manager before they ever come on site in a phone screen, 15-20 minute phone screen.

I urge you guys to do that. It's an exploratory conversation with the candidate. You describe the job a little bit. You ask the candidate what they've done. You give a short version of the most significant accomplishment question.

Then, you invite the candidate, "Hey, I really like your background. Would you like to come on site?" That's the way it would work. It's just this nurturing process. If you do it slowly, it can still be done. But it takes a lot of time, hours, at least to understand, "Hey, this is a career move."

If you do that, number one, you never have to meet people who...it's a waste of time to meet anybody in person, who's not at least in the top third, a real possible hire. It's just a waste of time.

At least this way, you meet a few more people on the phone and to record what he or she is doing. You meet some good candidates. Then, they push the apply button. Yes, I do want to apply, because there are some legal requirements when they apply for a job.

Nonetheless, there's no legal...let's say this. You have a lot of leeway, with respect to legal aspects of candidate movement and evaluation. Before they apply, you have some rigid rules once they apply for the job. This is the process.

It's a little slower process, spread over a couple of weeks. But it starts by, "Do you have a career move?" If you don't have a career move, you have nothing. That's what performance-based hiring is.

In that little card, it's totally summarized. Let me give you the quick take. A performance profile is the performance-based job description. You have to know what he work is. You have to know what the work is, in terms of outcomes.

Every job can be represented by four or five key performance objectives. Number two, talent centric sourcing is going after everybody in the total market. The 99 percent that don't apply, how do you get those people? It's better advertising, better networking, and nurturing with the recruiter and a hiring manager.

The third step is the interview. The hard core of the interview is what I've just done here with you guys, asking the most significant accomplishment question. Not again to the other part. You get the book. I also get into the work history of you.

Again, I'll ask some questions in a few minutes here. You can ask some questions in a few minutes. Whoever asks some good questions gets a copy of the book, and then you'll hire five or six good people next year. Then, you got to close the deal. I don't close the deal on money.

I close the deal on this, what you guys should do. Three or four days, or a week before you want to make a candidate an offer, either you or the recruiter asks these questions.

"Let's put our compensation package in the parking lot. I know it's important, but let's put it in the parking lot. Do you want this job?" You listen to the candidate.

If the candidate can't tell you why he or she wants this job at the internal gut level at the intrinsic motivator level, you're hiring the wrong person. The person will underperform, because they're only taking it for the compensation package.

They got to not only sell themselves. They got to sell their family, co-workers, friends, other recruiters, other person. They're going to offer more money. If you can sell them in their long-term intrinsic motivator, you're going to hire a great person.

If you sell them on a short-term what they get in the compensation package, it's problematic if you hired a good person. Works now and then, but it's pure luck. It wasn't planned. Performance-based hiring is a business process for hiring not a point solution.

Interviewing is a point solution. If candidates don't want to be interviewed, it's a useless solution. Each one of those things separately is a point solution. Putting it together, you have a business process. That's the idea.

There's four pillars to what I would consider an exceptional business process for hiring to pull it off. It's you the manager. I don't care all the complaints you have. I worked with Chrome companies that have great managers, they hire great people, because that's the domino thing.

Candidates will accept jobs for managers. Number two, you have the right strategy. As far as I'm concerned, hiring manager has to describe a strategy, if you're at the HR. Recruiting department won't. Most important of all is the job itself. You might be a great manager.

If you can't clarify expectations upfront and tell people what they're going to do, the likelihood of hiring the right person is problematic. Who knows? You got to tell people what the job is, at least the top four or five performance objectives.

The process has to be focused on a scarcity of talent, respectful process. The recruiter better know your job. Those four pillars, they're all different balances. They all have to be in place, to hire great people every time. Let me open up the questions, then I'll give a quick summary.
 
We got a few minutes for our questions. Remember, if it's a good question, you get a book. Jared would be responsible for asking, because he pre-screened these questions. I might go a little bit less structured here. Which is the first one, Jared?
Jared: Where Joan Willis? Right there.
Joan Willis: Good morning. How do you recommend we work with our HR Departments that tend to be more compliance and legal-driven, who put us in situations where we're going into the very static, descriptive...
Lou: I can understand that. I apologize for cutting you off. I know we don't have a few minutes. I can see this clock going down here. If you are the hiring manager, you get a lot of control.

First of all, you'll get a copy of the book. You'll turn to page, whatever it is. It's page 192. You see a white paper from Luke Mendelssohn and David Goldstein, the number one labor attorney in the US.
 
He says, "What you're doing today is fundamentally flawed." Luke Mendelssohn says, "We could do this. I'd like to try it." That's how I would do it. Just try if you want. We always do a try in a little bit of pilot basis. That's how I would overcome it.
Audience Member: Good morning, I have question about company's top talent programs. I find now, especially in upper mid-level management. Folks really want to know what is your top talent program.

How are you going to nurture me? How are you going to get me to the next level? They're very comfortable having those conversations, more comfortable than the HR Department are.
Lou: These are candidates asking these questions?
Audience Member: Uh-huh.
Lou: I think if you really clearly understand what the job is -- hey, you're going to be responsible for launching a new product line.

The key thing is, if you pull that off, obviously you're going to get bigger projects and more success. It really hinges upon we need you to launch a new product line, take over 10 percent of the Northeast territory, and get this thing globally launched within 9 months.

You pull that off, opportunities are going to be galore because here we're growing at 15 percent, and your company's growing at 5 percent. The reason they're asking those questions is because you obviously haven't clarified the work they're going to be doing. It's hard to believe.
 
I've talked to so many people that they are confused. They're missing something in the equation where they have to ask that question. I think it's a very fair question, but I suspect it's because the work that they're going to be doing isn't fully clarified. That would be my guess. I would at least try that.
Jared: What does Rob say? Oh they you are. Mark's next, and then Robin.
Mark: If you close a candidate on the vision and outcome of the performance side of things only to deliver them a substandard compensation package because you've put it in a parking lot, isn't that a waste of a lot of their time and for customer experience?
Lou: No, I don't think I said that. I think you said it. I never said substandard. I first said, as long as you're above a threshold of equitable fairness which is probably at the 50 percent or 60 percent level, if you're in the lower third you have a challenge.

There's no question about it. You have to be very honest about it, but I said here's the compensation package. Do you want the job irrespective of the compensation package?

If they can't justify that they want the job, then I wouldn't hire them. It is certainly the job and the compensation package together, but I want to separate those two things.
Mark: Thank you. I missed those specific two words.
Lou: Now you've got them, yeah.
Rob: Hi, Lou. I know technology doesn't replace...
Lou: Wait, who's talking? I can't see them.
Jared: I can.
Lou: Oh, thank you. [laughs]
Audience: [laughs]
Lou: I hear these voices coming, and I can't see where.
Rob: This is Rob.
Lou: They're very weird here, I tell you.
Rob: I know technology doesn't replace relationships between recruiters and candidates. Yet recruiters often have to find candidates through key word searches and things like that in their applicant tracking system.

My question is, what does a great performance-based applicant tracking system look like then?
Lou: That's a fair question. I can only tell you that I'm working with two of them, but I don't know that they are great -- Greenhouse and lever.company.

They allow folks to actually not use traditional job descriptions. They can have the internal job description. They recognize it doesn't have to be the external job description.

No company actually takes their product specifications and publishes them for marketing purposes. You've got to be an idiot to do that. If there's any HR people here, why would you do that? Draw a little picture of that [inaudible 55:45] , whatever it is, it's those kinds of things.

I think those ATS systems that offer that kind of flexibility, different titles, different structure, different ways of communicating amongst all the team. We get into the spec details, but I was doing them yesterday.
 
That's why I know those issues. Hopefully, that's a fair question. Any other questions? We can go more. I only have four books though, so you guys will get them -- just four.
Jared: Julie Mont? Where's Julie? Over there.
Lou: If you ask a good question, give me your card and I'll send you a book -- if it's a good question.
Julie Mont: What are your thoughts on portfolios? I know a lot of jobs require that you input a portfolio, but oftentimes the work that we've done in the past...I'm a user space researcher. Most of it's internal use only, so it's hard to actually visually show anything in a portfolio but maybe it can be described.
Lou: There I'd say I like to actually see samples of the person's work. Actually, that is a good predictor of on the job performance, the sample. I've had people look at financial reports and do some kind of analysis.

I've had people bring in products. I had one guy actually wheeled in almost a wheelbarrow, which I think was a little bit over the top.

 [laughter]

This box he built. I said, "Oh, God," the fact that he would do that literally. I said, "Bring in a sample." I was hoping it would be small, and something. I can't answer the question exactly.

If there was a way you could relate the complexity, the scale of the scope and the design capability with something that you've done to a physical design, I think you could get there.

It would be hard for me, I'd have to actually see the job itself and why there wouldn't be some kind of physical work that could be done. But basically, that's what I'm really try to do is trying to find out, seeing what physical product has this person delivered that's comparable to what I need done?

I know that was probably not a satisfactory answer, but I've got 46 seconds left so I had to go with it.

 [laughter]

Guys, I think that little card -- I know I was being a little bit facetious about it, but if you follow that card and you build a performance-based job description, you tell your recruiters what they're looking for. You reach out and do a lot of networking, you will actually be able to hire great people every time.

You will be able to call me a year from now and say, "Lou, thank you very much for hiring much for hiring great people." Thank you very much for today. Thank you, everybody.

 [applause]

About the Conference

UX Advantage covers UX strategy issues no one else is talking about.

Stay In Touch

Stay up-to-date about the next UX Advantage conference, live recordings, and podcasts.

UIE values your privacy.
We will never sell your e-mail address.