2021 IT Annual Report
Coming Back–in Style
Led by Audio-Visual Technologies, the IT staff helped Pepperdine faculty transition back to on-ground instruction using old-school work ethic and classroom technology.
Table of Contents
See about IT
The More Things Change
“Once more unto the breach, dear friends.”
No one will ever hear me claim to be an expert on Shakespeare, but this famous line from his play, Henry V, effectively sums up our approach to returning to on-ground instruction at Pepperdine’s South-ern California campuses on June 7, 2021. Simply put: This has been challenging, but we need to move forward.
Last spring, the University made the decision to return to in-person classes after a trying 15-plus months in COVID-19 lockdown. Students, faculty, and staff have been battling all the elements of remote work and instruction including time management, technological challenges, massive shifts in communication practices, and for many, an unimaginable fracture in our daily interpersonal relationships.
Yet, through University programs such as the Student Care Team and COVID-19 Vaccine Clinics, along with a dedicated faculty, new classroom technology, and technical support from IT, the Pepperdine community is adjusting to these paradigm shifts in every aspect of our lives. Strength of character and pure resilience are the unquantified difference in successfully transitioning from where we were at any point in our lives to where we strive to be. And where we strive to be at Pepperdine is together.
Born of our community’s unwavering spirit and determination, we drive forward into 2022 with the confidence that we can continue to adapt, to learn, and to persevere through our collaboration and confidence in one another. Or as Shakespeare said, “The game’s afoot: Follow your spirit…”
Working their way Up
Three graduating seniors in computer science at Pepperdine took a big step toward beginning their careers by earning internships on the IT department’s Innovative Development team in the 2021-22 academic year.
Pepperdine University’s IT department is known for many things in the community, from 24/7 technical support, to faculty assistance in the classrooms, to keeping our data secure. But not everyone knows about a potentially life-changing opportunity for students that IT offers every academic year: a paid internship on the Innovative Development team.
“We get our list of prospective juniors and seniors from Professor Stan Warford in Seaver College’s Natural Science Division,” said IT Director of Innovative Development Tim Bodden. “He heads up the computer science program, but we have had a few people in the past who came from other programs. The internship can also count as their capstone project.”
For 2021-22, there are three interns—Catherine Munzar, Ira Porchia, Austin Zell—who are all seniors. This year, all three are focused on a group project for a University client.
“We’re working on a web application for Pepperdine POC (Planning Operations, and Construction),” said senior Austin Zell. “It uses an existing database that they have for their codes, projects, and operations. We’re working to convert it into a web application, so it’s easier for them to use with the web interface.
“It’s currently in a Microsoft spreadsheet that they’ve been using for a long time, and they just want to transition into something more modern,” Zell continued.
“Every week, we have one meeting with them that ranges anywhere from 15 to 30 minutes,” said Zell. “We have weekly updates and discuss anything we may have questions on, and they’re also readily available outside of normally scheduled times. They’re there to help us, and the relationship is really friendly. It’s not just a work relationship: it can definitely expand into a mentorship or even a friendship. It’s a really nice dynamic.”
“I’ve been able to set up multiple meetings with Danny over Google Meet on a weekly basis,” noted Porchia about IT Senior Lead Developer Daniil Gedgafov. “And whenever he’s busy, I’m able to just send him a Google Chat, and he’ll respond right away. Without him, I’d still be stuck on the first part of the project.”
“Honestly, I was really intimidated at first when they said we would have mentors, because I thought they would be telling us everything we did wrong,” said Munzar. “But for me, it’s been really refreshing to look forward to the meetings, because it’s nice to have someone with experience look over what you’re doing and validate it.”
In this time of social distancing and remote work, the students also have the option of working remotely, but a funny thing happened on the way to the internship this year: they prefer the in-person experience.
“They actually enjoy coming to the office to get away from people,” Bodden said. “They need some quiet time to work on things.”
“It was also the second time we’ve ever done a group internship,” Bodden continued. “Group internships tend to work really well when it comes to them having to learn to work with others and handle project management issues.”
With just a few months left before they graduate, how do the interns feel now about hitting the job market?
“The academic front of it is purely fundamental,” said Zell of the experience. “It’s the concepts that you need to build off in your future work experiences. And this internship really gives you that real-world application in the workforce: that experience that you need to understand what interviewers will be asking you for in the future, and the technologies that they don’t introduce you to in class.”
“I felt like it’s definitely a lot more practical and the internship prepared all of us for the next chapter of our lives,” Porchia added. “This definitely puts me over the top in an interview.”
“Before, we were just doing little coding assignments or making something loop,” Munzar noted. “But now, it’s actually connected. It’s just a less intimidating way to enter the workforce.”
“It’s a good experience because it helps them with placement,” Bodden concluded. “It gives them something to put on their resume–they have a project that they can show off in an interview. They can say that they actually worked on that.”
Called to Service
When Pepperdine University returned to in-person classes, Information Technology staff provided technical support from the phones to the classrooms to ease the transition for the faculty.
As the pandemic continues to linger, it tends to push our communities apart, making many of us nostalgic for earlier times when it seemed easier to extend a friendly hand to someone in need. Few in the Pepperdine community who experienced the innumerable trials that came with the pandemic and the challenges in returning to on-ground instruction in 2021 are likely to remember recent times as happy days.
Yet, when Pepperdine brought the students back on-ground for a summer session on June 7, the Information Technology (IT) staff rolled up their collective sleeves and relied on a renewed sense of cooperation and a tenacious work ethic to turn the tide and make the transition as smooth as possible for our faculty.
“We had what I call the ‘COVID diaspora’ for a year and a half,” said Senior Director of IT Administration and Client Services Gerard Flynn. “Faculty hadn’t been in the classroom, and students hadn’t been back on campus, so we had a number of preparations to accommodate various student and faculty needs in their return.
“Some international students couldn’t return to the United States and yet still remained enrolled, other students had to be quarantined and yet still had to ‘attend’ class; and we had to prepare our campuses under the existing COVID protocols,” Flynn added. “So, we brought new equipment into every classroom to allow faculty to conduct Zoom sessions. We installed cameras in the back, provided lapel mics, and added what we call a puck mic to capture room audio in some locations.”
The new equipment was just the first crucial step in preparing for on-ground instruction. IT Training created instructional videos, and Technology & Learning worked with IT Communications to document processes and provide faculty resources on the IT website.
“We had worked on a series of short instructional videos in collaboration with groups at the University to help faculty with any questions or issues with classroom technology,” said Manager of IT Training and Technology & Learning Jordan Lott. “The videos ranged from two to 13 minutes on specific topics like Introduction to Classroom AV, How to Use the Long-Distance AV Equipment, and Connecting to the AV System and Troubleshooting.”
Senior Manager of Client Services Jared Mukai also noted that while the task may have seemed daunting to many faculty members, the project was designed to help them master the hybrid classroom. Just to be certain that his team was addressing every possible need for technical training and support for every member of the faculty, Mukai did the easiest thing he could: he asked them outright by sending a survey before the term.
“We wanted to get ahead of it,” said Mukai. “We thought, ‘You tell us: What do you need to know…what do you need from us?’ And we’ll do the best we can. We were able to address specific concerns and help a good number of people that way.”
Mukai and Flynn then worked together to build a project they called Blitz Time, referring to the IT classroom support effort for the first two weeks of the summer session beginning on June 7. IT relied on the Audio Visual Technologies (AVT) team of Sean Kalaras, Jordan Seah, Ramela Tehrani, Cale Watson, and Nick Baker to blanket the classrooms with technical support. But even with AVT’s expertise, there were obstacles to overcome.
“We had to make sure that the 50-some faculty that were going to be teaching in that summer session had more than a fighting chance to use the new hybrid equipment that we installed in 2020,” said Mukai. “And I think the biggest hurdle to all of that was just the uncertainty. Faculty didn’t know what to expect. They didn’t know what they were supposed to do. They didn’t know what walking into the classroom was going to be like.”
Next, Mukai decided to bring in reinforcements to beef up available technical support.
“Client Services saw this as an all-hands-on-deck sort of situation,” Mukai continued, “so we conscripted as many IT colleagues as we could and assigned each to two or three faculty. So, for every day and every class in June, faculty had an IT expert in the hallways of every building who could help them.”
At times, IT staff simply called an AVT colleague or Tech Central on a faculty member’s behalf to get the ball rolling. Quick solutions were always the goal, but timely communication drove the project’s success. And the faculty responded in kind.
“Just a quick note to express my sincere gratitude for DeJuan Oliver’s excellent tech support this morning, last week, and of course, throughout last year,” said Assistant Professor of Hispanic Studies Dr. Alison Stewart. “I was able to learn with relative ease this morning how to get a Zoom recording up and going in less than 15 minutes in my classroom! DeJuan showed me step by step how to do so and explained everything very clearly,” Professor Stewart continued. “Many thanks to your office for all of your help and notably to DeJuan for his excellent support during a very challenging transitional time!”
“I just wanted to let you know how GRATEFUL I am for the incredible tech support I have seen and experienced on these first two (chaotic) days of class,” said Associate Professor of Sports Medicine Dr. Cooker Storm. “I appreciate the thoughtfulness of your team to have help at-the-ready for every single class in every room; it has really been so impressive.”
Mukai used the same strategy for the second summer term in July, sending all faculty a survey to determine their technological needs and questions, while scheduling IT support staff for the first two weeks of the session.
“The feedback we got from our IT staff in the field was that after the first two weeks, faculty didn’t really need us,” Mukai said, “which we thought was great! We couldn’t have asked for a better piece of feedback.
“There were about 50 faculty for June, and 30 for July, but the fall semester was going to be the big one,” Mukai noted. “That meant approximately 900 faculty members across all our campuses, with more than 600 in Malibu. And all the classrooms were going to be used at virtually 100 percent capacity, beginning at 8 AM and running until 10 PM.
“So, we did a scaled-up version of our summer approach by sending surveys to almost 650 faculty,” Mukai explained. “We asked if they had any questions, concerns, or special requests technology-wise that we could help them with.
“We also offered self-paced training videos along with 32 in-person training sessions at all the campuses,” Mukai said. “We offered to show our faculty the technology firsthand, answer questions, and provide demonstrations.
Mukai’s team complemented the hands-on training by checking the equipment in every classroom before classes began.
“We rounded out our preparation by checking the systems, making sure everything turned on, verifying every connection, even making sure all the lights worked…” Mukai recalled. “Those were the three things that we did to ensure our faculty’s success: the survey to provide them the opportunity to request any advance assistance or instruction; the in-person and video training offerings; and the robust proactive maintenance we conducted on all the equipment.”
When August 30 arrived, the IT department had more than 40 staff assigned to various academic buildings to answer any faculty questions and assist in basic classroom technology issues.
“From 7:30 in the morning until 10 o’clock every night, we rotated people in and out to handle all the survey requests, so that each had a dedicated IT person to handle their request and get them started in the best way possible,” Mukai explained.
“We were on-site to answer any questions people had while maintaining a command center in TAC (Thornton Administrative Center),” Mukai continued. “There we had staff answering the new 4449 phone number for just-in-time classroom AV support, so we could dispatch technicians as quickly as possible to get people the help they needed.
“Once the faculty didn’t require that level of support, we focused on transitioning to a sustainable, long-term support model,” Mukai said. “We had to create a system that allowed a five-person team to support 900 classes in Malibu, while also helping the technicians at each of the grad campuses.
“The takeaway here is around this idea that faculty need all this help in order for them to succeed during the semester,” said Mukai. “That’s just not true.
“But, if we can target who is calling us, what these calls are about, and where the call is coming from, there’s an opportunity for us to see if we can eliminate some of those pain points, improve the overall classroom experience, and reduce the need for emergency calls,” Mukai added.
“We’re especially pleased to find out that support requests tapered off throughout the semester, especially the ones in the early morning and in the evening,” said Flynn. “The new equipment is much easier to use and more reliable. Faculty also had more practice and could solve minor issues themselves. And I believe the proactive training we offered people really put them in good stead to be able to use the classroom technology.”
“We were over-prepared, which was our intention,” added Mukai. “We overstaffed. We over-prepared. We double- and triple-checked the equipment.”
With more than 1,200 combined survey responses and calls to the just-in-time number, Mukai now has the data he needs to drive faculty support levels..
“We’ve been able to optimize these processes so whether it’s the first day of school or the last day of finals,” Mukai noted, “the team that we have in place does have the bandwidth and the ability to provide the necessary support.
“It turns out that over half of callers–53%–have only called once during the semester,” Mukai continued. “So, either they called, had a small problem, and we fixed it, or they learned and it never came up again. In fact, 94% of all the callers have called five times or fewer over the whole semester.
“We now have a much clearer idea of what to expect,” Mukai continued. “Going forward, having this data allows us to plan more accurately for the upcoming need. The second thing is that we learned that the way we design the plan, the surveys, the web resources, the training, and even sending technicians to your first day of class are all scalable processes.
“One of the biggest goals with this new classroom support model is to be data driven,” noted Mukai. “This is something we’ve never had before, so we’ve never been able to leverage the benefits and advantages that clean, consistent, well-structured data can provide. I can now quantify these things to benefit our faculty.
“I can tell you how many times people will call,” Mukai continued. “I can tell you which faculty need the most help. I can tell you which building needs the most help. I can tell you what times of the day specific buildings need more support. I can tell you what the most common problem is. I can tell you what the top five most common problems are, and I can tell you that faculty can easily fix all five of them.
“We’re going to take care of everybody,” Mukai said. “It’s going to be fine. We can do this with a five-person AVT team. The data provides us insights to support our faculty that we’ve just never had,” Mukai said.
With IT’s data-driven approach, Jared Mukai’s team will deliver unparalleled technical support for remote and hybrid instruction across the full spectrum of Pepperdine classrooms. No need to get nostalgic: looks like the good old days aren’t that far away.
Log Jammed
On a quiet Friday afternoon in December 2021, Pepperdine’s Server Engineering team notified the Information Security Office (ISO) of some unusual activity on university servers, and the ISO team determined that it matched a global digital security attack: the Log4Shell exploit.
“Things started when Systems and Data Administrator Lance Coert found that there was a vulnerability with the eSports Minecraft server,” said Pepperdine University Chief Information Security Officer Kim Cary. “And when Systems and Data Administrator Canon Hamlin notified the Information Security Office he was mitigating it, I discovered that the problem was part of the larger vulnerability in the ubiquitous software plug-in Log4j that had been developing into the worldwide Log4Shell exploit that morning.”
The Log4j plug-in is heavily used in internet server installations worldwide. The ISO team reached out to other universities to learn how they were dealing with the same issue. “I began immediately by advising Pepperdine’s IT leadership that we might have an emergency issue,” Cary said. After quickly pulling together the necessary IT staff to scope the problem, Cary followed the IT emergency response communications protocols.
“And at about 2:30 on Friday afternoon, we notified our emergency response team to join the call because we had enough information to start acting on it across the University at that point. We had a working Zoom meeting with 30 Pepperdine staff,” said Cary. “ISO was also online with our colleagues at other universities who were dealing with the same exploit. As decision points would arise in the meeting, we could compare them internally and externally. That helped inform us to be as efficient and as secure as possible by being part of the larger community.
“Once we apprised Chief Information Officer (CIO) Jonathan See of these things, he began briefing University leadership, which left us free to deal with the nuts and bolts,” Cary explained. “Everything came together so well that the subsequent bulletin to the Pepperdine community was drafted by IT Communications, edited by two University vice presidents, and approved by the CIO by early evening.”
Cary called the IT department’s Log4Shell response an important collaboration with University leadership through the crisis. Multiple IT teams from Server Engineering to Information Security worked on the problem over that December weekend, and all production systems were back up on Sunday at 3 PM.
“We’re going to be dealing with the cleanup of this for months,” Cary said. “The vendors who were on top of this vulnerable plug-in have already provided fixes; for the rest, IT has turned off or protected the system until the vendor can get us the fixed software. That will take a while, but in the meantime we are secure, and monitoring network traffic daily to make sure nothing has gotten by us with the Log4j vulnerability.”
So, how did a medium-sized Christian University with a small Information Security Office manage to keep the Pepperdine community safe when the Log4j vulnerability had given hackers the ability to target computers all over the world? It didn’t happen overnight.
“IT has been on an upward trajectory since 2007,” Cary said. “One of the reasons why we’re not in trouble on this vulnerability is that we have proactive, professional IT teams.”
Cary also pointed to the IT department’s rigorous adherence to best practices and the regular updates across all systems.
“Whenever there’s a security update, we implement it immediately,” Cary said. “The server team and network team have been doing this for years in coordination with ISO. Additionally, when Pepperdine IT replaced the firewalls and intrusion detection systems in 2015 to allow for a faster network, we installed a next-generation firewall with better attack detection and automated defenses. So, we have very good IT housekeeping here at Pepperdine, and this put us in a strong defensive position.
“We also built an architecture around this next-generation firewall which allowed us to see some things that are hidden to other IT teams at other schools that I’ve spoken to,” Cary continued. “Consequently, the threat intel that the firewall manufacturer provided was more effective because we could actually see into our systems in a way that other institutions couldn’t.”
Cary detailed the explanation.
“Basically, encryption is our best friend and it’s our worst enemy when we are attacked,” he explained. “When you encrypt your network traffic, nobody can see it, which is good, including you, which is bad. But we have inbound decryption on key systems, so when somebody encrypts an inbound attack to us, our firewall can decode it and stop it. And that’s what the firewall was doing as the Log4Shell attacks began to mount up starting that Friday.”
Ultimately, while the Log4j vulnerability was wreaking havoc on the rest of the world, the Pepperdine IT department’s rigorous adherence to best practices and timely updates across all systems—including all those WaveNet upgrades Pepperdine knows so well—put the team in a position to be able to analyze and then successfully defend our community’s digital security.
Dashboard: 2021 By the Numbers
We hope you find the following data points from the calendar year 2021 as interesting as we did!
- SecureConnect: Student enrollment in SecureConnect, our multi-factor authentication service, increased 1558%.
- Time of Your Life: The cumulative time of all of the 2021 Zoom meetings and webinars adds up to 217.14 Years.
- Booked Solid: Our community scheduled 271,515 meeting or webinar events.
- Inbox Defense: Our University spam filters blocked 8.7 million spam or harmful messages while delivering 15.1 million messages.
- PeopleSoft Processes: Tallying some WaveNet stats, there were 92,957 class signups, 26,458 applications to our academic programs, and 11,828 enrolled students.
- Data Dictionary and Analytics: There are 295 data definitions and we saw an increase of 38% in distinct logins.
- Staff Training: We offered 142 real-time professional development classes with 541 total signups.
- LinkedIn Learning: Our LinkedIn Learning service that is available to all current students, faculty, and staff members had 177,972 video views.
- The Best Surfing: The total combined IT website visits accounted for more than 40% of all traffic on Pepperdine Community websites.
- Tech Central Tickets: Our Tech Central team closed 14,369 support requests for Pepperdine's faculty, staff, student, and applicant community members.
- Document Management: Approximately 264,311 documents were uploaded to our imaging service, Etrieve.
- Courses Logins: In 2021, we saw an increase of 9% in Courses logins or 2,765,663 total.