On this livestream episode of ACEDS #CaseoftheWeek, Kelly Twigger talks about the cost for recovery for TAR expenses the Lawson v. Spirit Aerosystems, Inc. 2020 WL 6343292 (D. Kan. 2020) presided over by United States Magistrate Judge Angel D. Mitchell.
Greetings, eDiscovery enthusiasts, and welcome to the ACEDS case of the week for December 8, 2020. I am Kelly Twigger, the CEO of eDiscovery Assistant and a principal at ESI attorneys. As you know from joining us each week, ACEDS and eDiscovery Assistant have teamed up to launch this livestream to help educate lawyers and legal professionals on eDiscovery case law. Our goal is to talk about the practical implications of decisions and give you some insights into how these cases can be applied in your practice.
As many of you know, eDiscovery Assistant maintains a curated database of eDiscovery decisions that are tagged by eDiscovery issues. Two of the issues that we are going to talk about today are tags in eDiscovery Assistant and those include Technology Assisted Review or TAR and cost recovery.
Let’s kind of talk a little bit about, before we get started, in the links in the comment section on our LinkedIn page. If you’re watching us on another platform, you want to pop over to the LinkedIn events page for this event today, and in there, you’ll see the link to this case from eDiscovery Assistant, which is available to be viewed publicly. You’ll also see two links from posts from Doug Austin, our good friend and partner or affiliate partner with eDiscovery Assistant over at his eDiscovery Today blog. Doug wrote, not only on the decision we’re going to discuss today, but also on one of the earlier decisions in Lawson vs. Spirit case.
You’re also going to see in there a now live link to register for the University of Florida eDiscovery Conference, which is going to be held on March 18th. I’m part of the planning committee for that conference, have been for years with Bill Hamilton and a number of really great colleagues and professionals in the eDiscovery space. We have a fantastic virtual event that we’re putting on for you this year; focuses entirely on practical skills and dealing with them in our virtual environment. The conference this year is going to be completely free. Click on that link. We’ve also shared it a couple of times on our eDiscovery Assistant LinkedIn page and Facebook page, and just sign up and then you’ll get the latest information about the UF event.
OK, so with all of that being said, let’s get into our case for this week and our decision this week is from the case Lawson vs. Spirit Aero Systems. This decision is from October 29, 2020. Again, another decision from Magistrate Judge Mitchell out of the United States District Court for the District of Kansas. Now, this decision is one of–wait for it–nine decisions on discovery in this case. This has been a very contentious discovery case. I have to say that if you’re teaching eDiscovery, you could practically use the decisions in the Lawson case as your entire course, as they cover just about every issue all the way from choosing custodians and search terms to definitively deciding on a TAR protocol and arguing about cost recovery. The cost recovery is where we’re going to look today.
But one of the things I wanted to show you quickly is that within eDiscovery Assistant, you can view each of those related decisions. When you click on the public link to the case, you’ll come to this view over here, I’ve kind of expanded it so you can see it a little better. This is the full text of the case. The way that we’ve set up eDiscovery Assistant is that you can see all the tagging structure for the case over here and the related rules for the case, but then you can also see additional decisions that are also on the same docket. I mentioned to you that there are eight other decisions. They are all included in eDiscovery Assistant so you can start at the bottom and work your way up.
All right, our issue on the ruling that we’re talking about today is really cost shifting for expenses related to TAR under Rule 26(c) of the Federal Rules of Civil Procedure.
What are the facts of our case? Lawson was the former CEO of Spirit Aero Systems, and he had a retirement agreement under which he was receiving benefits subsequent to leaving the company. Spirit stopped paying those retirement benefits based on some consulting that Lawson was doing with another company called Arconic, and Spirit claimed that Lawson breached his agreement by doing that consulting. Lawson sued to try and continue to receive his retirement benefits, and the entire dispute is about whether Spirit and Arconic’s businesses overlap such that consulting for Arconic was prohibited under Lawson’s retirement agreement.
The general theme of all of these decisions in the Lawson case is that Lawson was super aggressive about what he wanted to do in discovery. The court goes out of its way in most of the decisions, and in particular in the one that we’re discussing today to note that certain things were at Lawson’s request. As the court notes here, at Lawson’s request, party spent months on the discovery process and on the business overlap issues, starting first with identifying custodians for ESI and search terms to be used to identify responsive information.
Lawson was upset when those search terms and custodians did not reveal high responsiveness rates. He wanted to conduct technology assisted review and see if he could get better results that way. In the decision immediately prior to this decision, which again, you can view in eDiscovery Assistant, the court allowed the use of TAR, but it said subject to the court deciding whether it would allocate those TAR expenses to Lawson. You know from Rule 26 and the Federal Rules that cost shifting is permitted in instances where there’s an undue burden, and we look at the proportionality factors, there are also specific factors set out related to cost shifting.
That’s a whole other issue, another issue that’s tagged in eDiscovery Assistant and that we can evaluate on another case. In this decision today, the court had already decided that Lawson was going to bear the brunt of the cost. The question was how much? Following the TAR process and the production of materials under TAR, Spirit filed a motion to shift the cost to Lawson under Rule 26(c)(1)(b), and that section authorizes a court to allocate discovery expenses upon a showing of good cause in order to protect a party from undue burden or expense. Again, the proportionality discussion. The court granted that motion. We talked about that that’s the previous decision and allocated those TAR expenses for Lawson for insisting on pursuing TAR after it became disproportional to the needs of the case. This case is about where the court ordered briefing to consider the amounts, and that’s what we’re looking at today.
One of the things that we emphasize every week when we do this #CaseoftheWeek is we try to choose cases for you that are instructional, that are going to help you learn how these concepts are being applied by the courts in well authored decisions. I can tell you (having read these cases in Lawson multiple times, and I know Doug and my friend David Horrigan at Relativity we’ve all looked at these cases) they are so thoughtful; very well written by Judge Mitchell. They’re really worth your time in reading to highlight these issues. When you get to watching this video, click on the link and take a look through the case. This particular decision is long. None of them are particularly short, but they are well articulated. There’s not extra verbosity in them. They are good reading and good analysis to sit in your head as these issues go forward and you’re working on these in your practice.
Important to this particular case is that, after the parties had briefed the issues on cost, the parties also filed summary judgment motions. In the summary judgment motions–I guess I should back up for a second and tell you that there were a total of 322,524 documents that were in the TAR data set to begin with. I don’t know that the case mentions exactly how many of those 322,000 were eventually produced following the TAR process. What was noteworthy to the court is that in its summary judgment filing, Lawson listed 95 exhibits, but only one of them, just one, came from this entire TAR process. The important part about that is that that one exhibit was about an unremarkable factual contention about when Lawson began contemplating retirement and not about the issue of overlap between Spirit and Arconic that was the subject of the TAR and the reason that Lawson wanted to do it in the first place.
The results of a process are very telling to a court when you’re looking at who should bear the brunt of the expenses and whether or not they were proportional to the needs of the case.
What was the application for fees allowed? There were three large categories:
- A total of $455,000 that were paid to the service provider who conducted the TAR,
- A total of #340,000 in attorneys fees relative, including contract review for TAR, and
- Additional $83,000 in attorneys fees for the current application to the court.
$83,000 just to make the application to the court on the costs for fees. These kinds of applications are super time intensive. They take a lot of time, a lot of detail to put before the court. The court does a great job here in going through each category of costs and laying out in detail what recovery is about. The court starts with what the standard is, what is the standard for allowing cost recovery and what is the basis for cost recovery. It’s what’s reasonable in the circumstances. Reasonable, as you know, is our monicker in eDiscovery, and it can can be subjective. We are finding with the courts that are issuing very articulate decisions that they are looking at very objective facts to determine what would otherwise be a subjective consideration of what’s reasonable.
From a practical perspective, what that means is, is as you’re putting forward an application for cost, as you’re engaging in TAR, if you’re in a particularly contentious dispute, you should be documenting everything. And we’ll talk about that a little bit more in our practical takeaways.
Here, the court started with evaluating the service provider costs. I mentioned, there were a little over 322,000 documents in the data set by the service provider used Venio, which is an eDiscovery software, to de-duplicate extract the text and metadata for keyword searching. They then used an early case assessment to promote documents to the TAR tool with additional data was kept in a nearline state so they could promote additional data if that was necessary. Usually promotion to TAR tool is going to be based on the amount of data that you promote to that tool. It’s important to make sure that what you’re giving it is of value. Once the index was created of the TAR documents, the contract and law firm attorneys then reviewed to train the software, coding the software for not just responsiveness, but also for confidentiality and privilege. And that’s important.
If you watched last week’s case of the week, we talked about the costs associated with first past review and how a dollar, a document usually just gets you responsiveness determination and maybe an initial consideration of whether something might be privileged and possibly a confidentiality designation. A lot of times those are determined at a second second level of review. Those responsive documents were then subject to a second level of review by firm attorneys here.
The process that Spirit used was what we call TAR 2.0, also been called computer assisted learning. One of the arguments made by Lawson was that TAR 2.0 was too expensive, and he felt as though TAR 1.0 should have been used and would have cut the costs that Spirit sought on its application in half, roughly in half.
Let’s talk about a little bit, as the court does articulate the difference between TAR 1.0 and TAR 2.0, and essentially this is just an evolution of the technology. It’s a fantastic evolution.
As I told you last week when we talked about using TAR, this technology’s terrific, and it can really speed up the efficiency of your review and production. We’re going to have to have increases in technology to be able to continue to manage the volumes of ESI that we have in litigation. It’s just not sustainable from a cost perspective. It is really important for us as lawyers to start leveraging this technology and start learning how to use it effectively and start learning how to defend it in court.
How the judge’s articulation of the difference between TAR 1.0 and TAR 2.0 here in this Lawson decision is really important, and I think you should take some time to read it.
What are the differences? TAR 1.0 essentially required that we create outside of the TAR a seed set of documents, so we choose a random sampling of the set.
Of the 322,000, maybe you choose 10%. You take 32,000 documents and you’re going to code those documents for a variety of things to teach the software which documents are relevant. You then have that seed set in the TAR tool and the TAR tool creates a predictive algorithm. That predictive algorithm is then applied against the entire balance, the entire other 290,000 documents in that case. But it is a fixed algorithm. There’s no continued learning. If the same document, if there’s a duplicate document with a different hash value in the seed set and one in the other 290,000, and they are coded differently, the tool is not going to learn from that in TAR 1.0.
TAR 2.0 is what we call continuous active learning. And so that TAR 2.0 means that you you do human review and machine learning at exactly the same time. As I go in and start coding documents, the machine starts learning and it learns more and more with every document that’s coded. Then you continuously reapply that learning to the data set to rank documents in the importance of review. Usually we see them as percentage responsiveness from 100 all the way down to zero. Most often, if you’re familiar with the TAR process, will review down to a certain level of responsiveness and the parties can agree on that, or you can take a reasonable position–85% some places, 70% some places. Sometimes you go down to 70% and then you do a sampling review of what’s below 70% or between 70 and 40.
In any event, whatever you’re doing, whatever your process is, you want to make sure that you document it. In 2.0, again, it’s that ability to continually analyze the entire document collection and rank and re-rank the documents for relevancy. It’s a much more robust process now with TAR 2.0 than with TAR 1.0. Is it more expensive? That depends on the technology that you use and how knowledgeable the folks are that are using it. If you have folks who understand how to create an appropriate TAR protocol and run it and document it, you’re going to be more efficient than somebody who’s fumbling. There’s no question about that.
They did have competing experts from service providers who talked about the differences between TAR 2.0 and TAR 1.0. Ultimately, the court found that using the TAR 2.0 was reasonable. It was a productive use of the time. I think if you go back and look at, as I was reading this decision I was thinking, if we were reviewing this manually on a first past review, we would have spent about 322,000, not including any QC or any project management. When you look at the actual costs associated with that, the TAR process, ultimately you ended up with about total cost of an application of 455,000 here. Is that really different than you would have done with a linear reviewer with a TAR 1.0 process? My gut is that you might have saved a little bit of money or maybe you wouldn’t have. It really depends on how things are done and how the technology works. So it’s hard to determine, hard to make an apples to apples comparison here. What’s important is that the court found the use of TAR 2.0 to be reasonable. They also pointed to the fact that Lawson was involved in the early discussions about how TAR was going to be done and they never objected to using TAR 2.0. It’s difficult to make that argument on the flip side, when you didn’t object to it in the first place.
That’s a review of the court’s analysis on the reasonableness of costs for the service provider. With regard to the costs for review, the court looked at the review rates. What I want you to take away from this independent analysis from a court is that there are very specific facts that the court looked at here that you need to be able to put in your application. You need to keep those metrics as you’re going through this process. Here the court looked at the rate of review of 26 documents an hour, as well as the rates for review. I’m not going to specifically discuss those rates, although they are mentioned in the case. I think those of you who know or manage review, you know that rates differ in vary around the country and they vary for first past review as well as project management and QC. Those rates are going to vary.
The court looked at that rate of review of 26 documents an hour, as well as the cost per hour for each of the various people. And it looked at the experience rates that were being charged for each of those people. It found both the rate of review at 26 documents an hour and the rates that were being charged for those folks all to be reasonable. That’s an extreme level of detail. You can look at those rates and see that they have been held reasonable by a court.
One of the things that the court noted on the review rate of 26 documents an hour (if you recall from last week, we talked about there maybe being an average review rate of about 40 docs an hour on a first pass review), 26 docs an hour gave them not only responsiveness, but also confidentiality and privilege. That was more bang for your buck and less documents per hour, but less need or less time to be spent on a second pass review because that was done on the first pass review. The review costs were determined to be reasonable.
The next issue on the application was with regard to costs for creating privilege log for the TAR only. This is separate from the privilege log that was created related to the productions on the ESI and custodians. What the court found here is that generally the cost to create it were relevant or were reasonable. Got my terms wrong. But huge caveat, what the court did know is that some of the time entries from some of the attorneys were not specific as to whether or not their entries for creating the privilege log were for TAR or for the previous privilege log. They reduced them, I believe, by 50%. That’s a pretty significant amount. What’s the takeaway there? Got to be specific, when you’re doing an application for TAR expenses separate and apart from what you were looking for previously, you’ve got to make sure your time charges are going to reflect exactly what you need to be able to get that money on an application for cost.
The court also said that Lawson did need to bear the expenses associated with the privilege because that was another part of the TAR process that he demanded. They also reviewed and found that the project management rates and time that were charged were generally reasonable. They also reviewed the review hosting and production costs and elected to shift those to Lawson in part because of the 10 custodians that were used for TAR, Spirit had said that seven of them didn’t have any relevant ESI and they didn’t. It was reasonable for Lawson to pay for that, according to the court.
The court also reviewed the attorneys’ fees offer. They went into detail as courts are prone to do on rates. Who did what kind of work? There were some things that the court reduced the application for based on kind of looking at what partner rates were and how much work was done at a partner level and what rates were charged for that work. They dinged a few of the costs that were requested there. It is really important that you make sure you’re staffing appropriately and the rates you requesting for work are appropriate. Courts are paying a lot of attention to that.
What are our key takeaways from this Lawson decision?
I’ve mentioned it a couple of times, but in general, you want to look at what are the costs of doing TAR compared to a linear review that we talked about last week, and I’ve talked about this a couple of times. There are always going to be differences in the costs associated with the software. The key is to do your due diligence–understand how the software works that you’re using, what kind of data is in the data set that you’re applying to, what are the unique challenges of the review set, and what are the unique challenges of the algorithm being imposed upon your review set. If you have a lot of images and those images can’t be OCR’d for text, are you going to be able to teach the software on TAR to be able to record those images and to properly tag those images or properly rank those images, I should say. And if not, then is TAR going to be the the best bang for your buck or is TAR only going to be useful for part of your collection of documents. Understand your software. Understand your data that you’re applying it to. Then there’s a big piece that those of us who are engaged with the data all the time feel like is an issue, right? Everybody is concerned about cost. Everybody is concerned about price. Price is a number one consideration in looking at service providers, in looking at different technologies that we’re using. If price is the number one thing you look at, unless you are making a strict apples to apples comparison, which you rarely are, probably making a mistake and choosing solely based on price. You need to choose based on functionality and look at the total cost of what you’re using the technology for and the results you’ll get, as opposed to just pure cost going in.
Second, take away–document, document, document. We’ve talked about keeping track of just about everything that you do in the eDiscovery process however it is, even if at the end of the day, you just sit down and make bulleted notes about everything that happened that day and all the decisions that went into the process. That will allow you to be able to recreate that picture later; it’ll be very difficult to try and remember all of those decisions that were made if its not documented as you go. The more contentious the case, the more you need to track information. I can tell you, though, that part of our practice at ESI Attorneys is to track just about everything we do. Even in cases that are not contentious, we find that we have to go back and revisit that information very frequently.
You may sit on a case for six months or eight months following a summary judgment motion waiting for trial. You pick things up two months before trial and suddenly you have 20 questions that need to go back to what was done in the case originally. Your documentation process is really going to facilitate that. It’s going to going to make sure that you have the answers to the questions of what was done during discovery.
Next, take away. If you’re going to do an application for cost, the level of detail that’s going to be required to recover those costs is going to be very high. Make sure that you have all the detail you need. Make sure that you’re going to provide declarations from the appropriate parties who did the work and be able to justify what is reasonable because again, that’s your standard. This Lawson decision is the 9th one in the case; as we mentioned, it’s yet another excellent analysis by the court and understanding of the level of complexity and different skill sets that are required to complete the tasks. From a litigation support perspective to a paralegal perspective to the different levels of lawyers, the court really dug into what was important there, and you’ll need to provide that information on an application for to determine what’s reasonable.
Great decision from the court in Lawson. That is our case of the week. In cooperation with ACEDS, let me say thank you to our entire team for putting this program together and stay safe and have a great week.
Thanks so much.
Watch Next Episode Live
#CaseoftheWeek will be available on the ACEDS YouTube, Facebook, Twitter, and LinkedIn company pages. You’ll also be able to read episode summaries and watch the recorded show right here on the eDiscovery Assistant blog.
