Jason Tucker: [00:00:00] This is Dev branch, and today we’re talking about test driven development.
I’m Jason Tucker. You find me at Jason Tucker on Twitter. The same name can also be found over on Mastodon. This Sé Reed, she’s not here today. I couldn’t edit the video before I got this thing done.
Jason Cosper: And y’all know who it is. It’s your boy Jason Cosper, AKA Fat Mullenweg. Back at it again on the world’s most influential WordPress podcast.
Jason Tucker: Speaking of which, you can go over to Apple Podcast, Audible, Google Podcast, Spotify, and you can also hang out with us in Discord. Go over to our website@jpwatercore.com slash discord where you can find that information over there. Hello, How you doing everyone?
Steve Grunwell: Not too bad. How are you?
Jason Tucker: Going Good. Going good. we have a friend on the show.
Hey Steve, how are you doing today?[00:01:00]
I’m doing well. Thank you for having me.
Jason Tucker: Awesome. I’m gonna call you friend, even though we just met, it’s grown on me.
post formats, Malone and I go back several years,We worked together for four years. Steve, why don’t you introduce yourself.
Steve Grunwell: All right. hi, I . I’m Steve Grunwell. I’m a staff software engineer at MailChimp, but before that I spent five years at Liquid Web working on the managed WordPress and WooCommerce hosting platforms. I’ve been writing software for, oh, geez, a lot of my adult life at this point.
And, I am way too. I have a lot of opinions about automated testing, let’s put it that way, which is why y’all are having me on the show today.
Jason Cosper: Yeah. so I was listening to the, post status podcast, last week, the week before they had till Cruise on, and, Till was talking about. effectively, the fact that, with his Object Cash Pro plugin specifically because he built that from the ground [00:02:00] up and he built that with, tests and testing across so much of his code base, that effectively he only works about five to 10 minutes of support every day.
He’s like a lone developer and also does his own support. And, it was like, they talked about that a bunch on the post status podcast and I thought,Oh, like I could invite till on. And then I thought, I. I know somebody who has big feelings about tests, who was always pushing, like when, him and I worked together over at Web for, I was there for four years.
every bit of code, it was like, Hey, you should write some tests for that. and I was just like, Steve has given Word camp talks and everything else, so I thought,let’s get another perspective on this. Let’s, let’s talk to Steve and talk to, to, to him about the importance of,test driven development.
Steve Grunwell: That sounds great and honestly, I’m honored to be, considered in the same [00:03:00] breath as still. there’s definitely something to be said, to what he brought up. Around, testing, reducing that maintenance overhead because Yeah. a he had the luxury of, building. His plugin from scratch, so that it’s not a, Oh, we have to deal with writing tests around a nearly 20 year old application.
because that does complicate things tremendously. But yeah, once you have something where you’re saying, Hey, I have, a good sense of how everything works, how it all comes together, when a bug comes. Because in software bugs always will happen. you write a test before you fix it, then you fix it, and then you have something built in, running as part of your CI pipeline that if this test ever comes back, or this bug rather, you’re going to catch it because you already have a test for it.
this would be known as regression testing. and it has saved my bacon so many times.
Jason Cosper: Yeah, absolutely. now [00:04:00] I, you’ve given talks at Word camps, like I said about, test driven development and about how. people should start using it. especially,there, I know that there are tests that are written for WordPress, but so much of that is legacy code that is going on, 20 years old at this point.
Has the ship sailed. For, WordPress and test driven development is, that seems like an awful big lift, but, how do you think that can be like, improved in the WordPress world?
Steve Grunwell: So capital T, test driven development, is the idea of we’re writing the code or we’re writing the tests before we actually write the code. in that sense, yes, it sailed only because. The code has already been written. we can’t go back in time, write the tests before. Can we retroactively add tests?
Can we practice TDD as we write new code? Absolutely. but just if I may [00:05:00] give a high level overview of what that capital T DD would be, all capitals, Test driven development, the idea that we’re going to, write our automated tests to help describe how the thing is going to work. So when I give this function, this input, I expect this output.
Or if we’re doing something that’s more of an integration level test, when I do this, I expect these things to happen. This, it’s the input and the output and making sure that things are behaving the way we expect. By writing those tests first, it forces us to A, think through how we want things to actually happen.
But B, it helps encourage our design or influence our design, of the, whether it’s a function or a whole piece of functionality. We’re saying, okay, I need to make sure that perhaps I’m using dependency injection for this so that I can actually inject a test double in a test environment, as opposed to just like instantiating an object in here that we [00:06:00] have no way of controlling.
Knowing that getting these tests to pass, is the end goal. we are able to basically lay out the path we’re going to take. Then we take the path, and we verify that yes, all of our tests are passing, so we have satisfied. The need here. And this process is also, commonly referred to as red green refactor.
Steve Grunwell: You write the tests, they’re all going to fail because you’re testing stuff that doesn’t exist. So that would be red. you get the test to pass, so then it’s screen, and then you can go back and you can say, Okay, the way we approached this maybe wasn’t super optimal. how can we improve performance?
How can we, generally clean this up? We can then refactor the code with confidence that we’re not going to break things because all of those tests will continue to pass. And if our refactor suddenly breaks some of the tests, we know that,we’ve broken the functionality or that we wrote bad tests to begin with, which when you’re first starting off, testing, I’ll admit, may happen more often [00:07:00] than, you’d hope.
sure. Yeah. That.
Jason Tucker: that you’re testing something, knowing that it’s gonna break because of the fact that it doesn’t exist yet. that’s a good . That’s a good indication,
Steve Grunwell: bake an apple pie, you must first create the universe.
Jason Tucker: right.
first you need to scaffold the test, structure. then you need to write the test, then you get your apple pie at the end. and we could all use more pie in our lives, And sometimes you have to build the oven before you even make that pie, right?
Steve Grunwell: Absolutely.
Jason Tucker: It makes sure all the dependencies are good,
Jason Cosper: Steve,you and I used to work together,I love a metaphor. and,I’m sure that viewers know this as well. I, it’s not really a metaphor, but,and I hate to use them as an example, because I don’t agree with their business practices, but, I have heard that Amazon, before they launch a product, or actually before they even start a product, they write the press release for the product before.
They actually put it out. they basically go, Here’s our target, here’s where we’re looking [00:08:00] to go. And then they go ahead and build the product. And yeah, they may need to refactor the, the press release, at the end of the process because,maybe we didn’t hit things completely right.
But effectively, tell me if I’m wrong here. That’s what you’re doing is you’re writing. The expected outcome before you actually write the
when you phrase it like that, the expected outcome before we write the actual code. Yes. As far as the press release before I take issue with that, because generally we, we write tests and then we don’t go, steal a bunch of IP from. Other companies? at least I don’t, I can’t speak for everybody.
ingredients are ingredients, right? It doesn’t matter how you actually get them.
copyright lawyers might, disagree with that, but yeah.
Jason Cosper: Sure. a, as far as, as, testing and development goes, I know that you have,You have, really,[00:09:00] been on like I said, I’ve seen your word camp talks. I’ve seen your, stuff, and we’ll make sure to get, some of that link that’s up on WordPress tv. the ones that you’ve given in the past.
at least some of them. Sometimes the camps I speak at are small enough that we get. The B team of cameras and they’re like, Ah, the recording just didn’t happen. but yes,
Tucker has run,run recordings at word camps of all different sizes. He knows exactly how that goes, as
Steve Grunwell: Yeah.
Jason Tucker: them get up before the end of the, the event.
Jason Cosper: sure no.
Jason Tucker: we’re lying. Okay, cool. on my watch,
Jason Cosper: Yeah. but really, How, since we said the ship is sailed on WordPress core and that, the ship is prob, the ship is also sailed on, the big focus, in WordPress right now. Gutenberg as well. they [00:10:00] didn’t really, start building. Gutenberg with any sort of TDD practices, either, like they, they were writing tests and everything else, and their coverage was probably, remarkably better than, how things were before with, with WordPress core where there was no testing at all while it was B2 and then et cetera.
but.
Jason Cosper: What do you think can be done to help, improve, testing as a whole, like in the WordPress community? do you think that, like it should at this point come from plugin developers who are, starting, a new plugin? something like that? Or do you think that,I don’t know.
I’m just interested in hearing your opinion.
Steve Grunwell: So I. The, there’s a difference between test driven development and having automated tests, and while the ship has sailed for [00:11:00] test driven development, there’s still plenty of opportunity for people to write more tests. the core test framework, the. The code that WordPress actually runs when we’re running through, hey, do, does the registering of custom post types work as expected?
Do these menus get rendered? All of that can be extended, and that’s, this is how I prefer to write desks for, WordPress plugins, for instance, because I’m actually loading an instance of WordPress. because there’s no better way to see if something’s going to work as expected within WordPress than, actually running it in WordPress.
A lot. I think there’s a disconnect between the larger testing community where you get some people who are like absolute purists, like no, a unit test must absolutely never have any connection to the database or anything outside of this. And,these strict lines. You have to be willing to blur those lines when it comes to testing within WordPress, WordPress is a tightly coupled application.
Steve Grunwell: So you are not going to really have the idea of a true [00:12:00] unit test because just by loading up WordPress, it’s already dealing with a lot. It requires the database connection. it’s setting a lot of things in motion, that are difficult to account for in a cure unit test. So instead we focus. Perhaps individual methods and saying, Okay, I know that this is going to happen.
maybe I need to introduce a test double. So filters can be really great for that. like the pre HTP request filter, I use that all the time, when testing WordPress because anytime you’re using, the WP http, api, say that three times fast. so WP Remote get WP Remote Post, those are all going through WP Remote request, which has this pre HTP request filter.
Steve Grunwell: We can short circuit the request, preventing it from actually calling out to a third party service, and inject results that we know,are what we’re expecting to see. So if I want to test you, [00:13:00] How does my plugin behave? If the third party service that it relies on, is throwing 500 errors, or the URL changed and it’s a 4 0 4, I can write a test that basically says, Hey, I’m going to call out to this and it’s going to give me a stubbed response.
Stubbed being one of the five types of test testable. That way, I’m not actually making HTP requests. So if you know the server connection is flaky, I don’t have to worry about,ah, this isn’t behaving. Or I write something to try to trigger an error on a third party service, and then they fix that error and suddenly my tests are broken.
we wanna try to isolate dependencies as much as possible in our tests, as well as, adhering to the rule of don’t test what you don’t own. So we are not testing. Within our plugins. We’re not testing WordPress core, we’re not testing third party services. We’re testing how we interact with WordPress core and how we interact with third party services.
so the, there’s an [00:14:00] important concept, referred to as system under tests or s u t, I don’t know if it’s pronounced SU or suit because, I’m painfully online and that tends to be a largely written medium. I say on a video podcast. Digress. and I’ve gotten way off, off track here, but as far as where the testing.
Jason Tucker: by the way,
Steve Grunwell: Oh, yes, I’m sure I’m the first person to ever get on a soapbox about something that they’re passionate about. getting back to your question though of, where should the testing really come from? it’s so many things in WordPress, it’s going to come from those core contributors and, whether that’s someone who, has commit access or someone.
Is really interested in testing and has bandwidth to take on, hey, here’s this feature. we have code coverage reports, and code coverage tells us, when we ran through the tests, here are lines of code that were executed and here are lines that weren’t. so if we have stuff that’s not getting executed during tests, we can safely assume that [00:15:00] code isn’t actually being covered by any tests.
so word. Does publish, code coverage reports and we can generate those ourselves as well. so being able to say, Here are areas of WordPress that we don’t have test coverage for. So I’m going to go in and I’m going to write tests for this that raises. It’s the, rising tide raises all ships, idea because if we are more confident in WordPress core and how it behaves, then plugins that rely on WordPress, which would be, let me check my, I don’t know, I’m checking my watch in this joke, but, all of them, rely on WordPress.
Steve Grunwell: So by making WordPress more solid, we end up making the plugins more solid. At the same time, plugin authors should absolutely be writing tests for their plugins, because then they’re going to be able to ensure that things are working as expected. And you can set up CI pipelines to say, Yes, I wanna run through all of these automated tests, but swap out the underlying [00:16:00] version of Fort Press.
This is one of the nice things about using that, WordPress. Test framework, because we’re able to say, I wanna ensure compatibility from say, WordPress five two up to current. So we could say, Great. Run through it on each version of WordPress in there across different versions of php. And if I start getting failures on PHP seven four and WordPress five six, I know, there’s, I know where I need to look. This goes back to what Till was saying about, by catching those bugs earlier, by saying, Okay, I know it fails under these conditions so I can add, special handling there. suddenly he doesn’t have to deal with, Hey, I have an issue with this. I’m getting an error. Just Okay, great.
what version of the plugin? What version of WordPress? What version of php? tell me all of the stuff about your environ. He can say, Hey, this showed up here in the CI pipeline. I can address it and then just not have to deal with support requests around it. [00:17:00] Testing. It’s fun.
Jason Tucker: it sounds like it.
Jason Cosper: It is fun. I you, you called out, specifically,the default, WordPress testing framework. Are there other frameworks that you, that you like? I know that there are people who, and I know, you also have, done work with, Lavo and things like that,and try to like, have been trying to bring.
a little bit of lal like into WordPress,and the way that, they’re writing their plugins, et cetera. are there any other frameworks that, that you like or think are worth a look? If you’re, like, you’ve done the WordPress testing framework, you’ve gotten your head around it.
Jason Cosper: Like anything, a little more advance.
Steve Grunwell: So the WordPress core test framework under the hood is using PHP unit. PHP unit is the defacto. test runner within the PHP community. and especially for people practicing tdd, unit tests and [00:18:00] integration tests tend to be the go-tos. we do have, there’s another school of thought and, that’s called behavior driven development.
it functions in a lot of the same ways, and I’m gonna get a lot of hate for that. but the. The general idea is instead of saying, Okay, when I call this function, I expect, this output. given this input, I expect this output. This is more, I’m going to set up a scenario. and. Within the TDD world, we tend to arrange our tests according to arrange, Act assert.
Steve Grunwell: So I’m going to set up the scenario. I’m going to set up, if I need to have some posts in here because I’m testing on a post, I’m going to do all of that. I’m going to arrange all of the dependencies of the test. I’m going to act upon the test. So I’m actually going to call that function that I’m testing, and then I’m going to make assertions about the result.
if I’m expecting to get two, I might say assert. Two. And the result that I get, bdd behavior driven development instead uses [00:19:00] given when, then as a construct. But those two are very similar because given when then, So given I am an administrator, on a single site press instance, when I try to create a new user, with email address, Jason Cosper dot com. Then this should be true. we should see a new user. So rather than being at that function level, we’re talking more from, outside looking in, and thinking more how the user is actually interacting with the application. at the end of the day, We’re accomplishing the same thing. We’re just taking a different route to get there.
Steve Grunwell: And the BDD framework, of choice is be hat. and if you’ve ever tried to scaffold tests through, W P cli, it tries to encourage beha, for writing tests. So beha is definitely a way you can go. There’s still some setup necessary. doesn’t magically [00:20:00] know, like given I am an administrator on a single WordPress instance, you end up writing a lot of matchers to say, Okay, so when I say I am an administrator, that means you have to create a user with role of administrator, beyond, The hat and PHP unit.
there is a tool called Pest. I haven’t worked with it myself. and I could be mistaken, but I wanna say it might use PHP unit under the hood. Please don’t quote me on that. but now it’s committed to video, so I hope I’m right. but it tries to take some of the principles from just, which is a Facebook maintained, JavaScript tests runner.
that’s the current hotness in JavaScript land. and. So those are really the main areas. you’re also going to find libraries, and this is where,Jason you mentioned, sorry, Cosper, you mentioned. I need to specify my Jason’s here, that I’ve been trying to [00:21:00] bring, some better stuff into WordPress testing. A lot of this will be done through traits that we can apply to PHP in it test classes. so I have, of course now I’m spacing on the names of the libraries. I’ll make sure. Send you the links so we can get ’em in the show notes. but libraries to basically help with PHP in it, to, if you need to do things that are going to work with, globals, because typically within tests, we don’t wanna actually manipulate global state.
unfortunately, WordPress,that’s its bread and butter manipulating global state. so I have a library. It’s, PHP unit test assertions. I think, or PHP in a global state. again, I’ll get you the links, but this library is actually a rapper around Run Kit, which is a php extension that actually lets you monkey patch the code or redefine like a function at runtime.
Steve Grunwell: So you could say, Hey, don’t do that thing. let me completely [00:22:00] redefine how you work. Or, Hey, we have these constants. If I need to test how this behaves, when a constant is said a certain way, but that constant is coming from WP config and I can’t change it, because that violates the nature of constants.
sometimes it’s handy to be able to. To manipulate, this within our tests. Of course, you never wanna use run kit in a production thing. And as I recall, the author of, Run Kit originally was like, Yeah, I got drunk one night and wrote a bad idea. and it became popular. So it stuck. Yeah. that’s how, let’s be honest, like 60% of software gets written, starts off a Oh yeah.
Steve Grunwell: I’ll show you. Oh no. What if I done, But in order to test a tightly coupled application that depends heavily on global state like WordPress, we have. Get through some of those, clear boundaries that if you were writing things from scratch, you would go, No, I’m not going to fall into those traps.
we’ve already fallen into those traps, partly because we didn’t have, [00:23:00] test driven development, helping to guide the design of these things. so sometimes we have to break the rules a little bit. It’s one of the, you have to understand the rules so you know when to break them. with global state and tests. Absolutely one of those cases. but, yeah, additional libraries are definitely out there, to aid in testing, especially within a WordPress context. Okay.
Jason Cosper: Yeah, for sure.
Jason Tucker: You think someone should begin who’s just a single, person, developer shop that’s building a plugin? how should they begin with, with this?
Steve Grunwell: One of my Covid projects was to put together a course, and I wish I had actually, followed through on that because this would be a great place for a plug. Unfortunately, I didn’t. so taking a look at, examples that are out there as far as testing goes, taking a look at the slides, from my confidently testing WordPress talk.
again, [00:24:00] I’ll make sure. I get those to you for the show notes. but getting a sense of how testing is meant to work, the idea of a system under test. The idea of making assertions, the way that we arrange our tests, can really have a huge impact on the end result. and just generally get used to the idea of. Thinking through, how am I going to test this? Even if you’re writing tests for something that’s already been written, thinking, okay, if I needed to set up a scenario so that I could actually run through this in an automated way, what would I need to do? what things need to be in place, What utilities might I need so that I’m not writing, 700 lines of code to set up a test.
Steve Grunwell: Ultimately, automated testing is meant. Make it much faster, to run through these things. We’re not sitting there manually clicking through everything over and over. it’s supposed to alert us when something breaks, that we’ve already written. and it lets us refactor our code,[00:25:00] with confidence because we have those guardrails set up.
I would say start with the slides. I owe the community some more writing on the topic clearly.
Jason Tucker: darn
Steve Grunwell: Yeah. definitely continue to listen to Good Dev Focus podcast like this one. Ooh, there’s a reverse plug I just plugged you on. You. and, reach out. ask questions, and find. Find happiness and find your zen in testing.
Jason Tucker: We’re okay with the moment of Zen
Jason Cosper: Yeah.
Jason Tucker: Steve, I want, I wanna say thank you very much for coming and hanging out with us. we really appreciate it and, I’m glad you’re able to,explain to us. I, I don’t know a whole lot about testing, so it was really great to hear this, especially in a structured environment of how you go about testing.
So thank you for that. And, links will be in the show notes like we’re talking about. So feel free to go take a look at those and hit up Steve [00:26:00] on. what, where should people hit you up on? If we’re going, If
don’t even know anymore. I’m still technically on Twitter, but I don’t use it. I’m at Steve Greenwell on, I used to start my docs with, I’m at Steve Greenwell on the social networks that matter, but that list is getting smaller and smaller. See one front
Steve Grunwell: yeah. steve conwell.com. The blog that doesn’t get updated.
Jason Tucker: That’s all right. That’s all right. There’ll be a course on there soon, so it’ll work out just fine.
Steve Grunwell: Fingers crossed.
good. Thank you very much, Steve. Here is our outro. You listen to the show or you watch the show, you can watch it or listen to it in various ways. On Apple Podcast, Google Podcast, Stitcher, Spotify, wherever it is, you listen to podcasts. You can also watch us on YouTube and Facebook and a few other places. We’re also on LinkedIn. Talk to y’all later.
Jason Tucker: Bye-bye.
Leave a Reply